Perhaps the only “general” ranking of minimum rigor/difficulty can be based on core or GE requirements, both in volume and rigor of such, since they are common to all students. Even then, some students may find some types of GE requirements more difficult than others, so the ranking may vary from one student to another. But a first approximation:
Caltech
MIT, Chicago, Columbia, any others with heavy core or GE requirements
all others somewhere in here
Amherst, Brown (no GE requirements)