Regarding rankings, Wesleyan in most years attracts a student body with at least as strong an academic profile as Middlebury’s. This analysis was created a few years ago and placed Wesleyan somewhat ahead of Middlebury, for example:
https://amp.businessinsider.com/the-610-smartest-colleges-in-america-2015-9
For a comparison pertaining to climate, people should note that Middetown’s winter temperatures fall closer to those of Nashville, Tennessee than to those of Middlebury, Vermont.
With respect to your academic programs of tentative interest, Middlebury is deservedly well known for environmental studies and languages. Wesleyan would be stronger for psychology, however.
In terms of your overall criteria and preferences, I think Wesleyan would be a stronger match for you. At the appropriate time, you could pursue an immersive environmental studies term to supplement your studies there: https://www.hamilton.edu/academics/offcampusstudy/adk.
Best of luck with your decision.