Caltech?
(If you also include required rigor that is not necessarily academic, you can add in places like the military service academies.)
However, more generally, the minimum academic rigor level in college does not necessarily increase as much as admission selectivity.