Bard College (Bard also has a conservatory, but there is a separate music program in the college) - Marka Gustavsson is a wonderful violist and teacher. Even if you don’t major in music, you can have lessons for free and be included in as many ensembles as you like. (My daughter did this, with majors in Photography and Literature.)
Bard has a Physics major with at least one Astrophysics course, but I doubt they have a full major in Astrophysics. HOWEVER, as a small school, they support students doing independent research and all students complete a senior thesis in their field. You could certainly choose to do your senior project on a topic in Astrophysics, which could set you up nicely for grad school.
Bard is about an hour by train north of New York City.