Middlebury, Wesleyan and Colgate would be among the best choices in the country for the study of economics:
https://ideas.repec.org/top/top.uslacecon.html
Brandeis would also be superb for economics.
Bowdoin appears in a PR sampling, “Great Schools for Mathematics Majors,” and would be a solid choice for study in this field.
Vassar would be academically competitive with the best from your group and would offer a less sports oriented option:
http://www.businessinsider.com/the-610-smartest-colleges-in-america-2015-9