Sounds like a recent Vassar grad that I know:
Came in as a prospective chem / math major and ended up majoring in math. He arrived as a STEM guy, but discovered philosophy and history of art (and was grateful to the writing center for helping him learn how to actually write a long form essay, something he had managed to avoid in HS). Made a great group of friends, got an amazing summer internship with the NSA (who sent him to the UK the following summer, to work with their counterparts at MI6!). He is now doing his PhD in math in a top 10 program, living with one of his college roommates and meeting up with the others semi-regularly. He said that the hardest part about adjusting to Vassar first semester was that he kept staying up too late at night b/c there were all these interesting people to talk to 
BUT: while I am a big fan of Vassar, for ED I strongly suggest that your son’s response to the school is the most important variable. It is a ‘fit’ school- and ime, kids know pretty quickly if they like the ‘feel’ of it or not. Move heaven and earth to take him to both Williams and Vassar, and trust his response.
Finally, I say this kindly, but the point is real:
“we” aren’t going to college- only your son is going. And if he is going to apply via ED, he needs to be sure for himself that he really wants to go there.