Smith is probably very well connected to Boston.
OP, making a big decision as a 17 or 18 year old seems daunting, for sure. But you have to trust your gut at some point. Face your fear and move forward. You won’t go wrong with either, for sure. You would probably be very happy at both schools. But, if Vassar is speaking to you, listen. Good luck!