I knew three people who did undergrad at MIT, and either graduate school or law school at Harvard. All three agree that MIT is significantly more difficult than Harvard academically.
I have degrees from MIT and Stanford. I would put them quite close to each other. The quarter system at Stanford will go a lot better if you never let yourself fall behind in any class – the end of the quarter comes up quickly. However, in terms of overall academic rigor I don’t think there is much difference. There is probably a larger difference between specific professors and specific classes within either school than between schools. If I had to choose I would put MIT as more academically tough than Stanford, but not by much.