So much is major dependent. And the specific courses taken even within a major. Columbia traditionally offered A+ grades for exceptional work - not sure if it still does - so that can up the perception of difficulty for the average student (just like at all the Ivies, the very best kids are in a totally different class from the average student there - and the A+ at Columbia recognized at least some of that).
All that being said, the answer to which elite school is the easiest is always the same: Brown 