I went to an exhibition (Sagmeister & Walsh: Beauty) where they dedicated some part to this. They claim that classical buildings are usually beautiful thanks to all the details and that modernist buildings are usually not so beutiful thanks to the missing details.

They also had this experiment where the visitors choose shapes and colours they find pretty. Rectangles and brown-red were the least favourite by far. And then they asked why do we build buildings out of the vomponents we find the most hideous?

----

I have my doubts whether their reasoning is valid, but it is somewhat relevant to your comment.