As a general consideration, physics tends to represent the more comprehensive and respected major for an undergraduate interested in astronomy/astrophysics. Choosing electives that would lead to an astronomy minor would balance this approach. If you might be interested in planetary science, you would benefit from geosciences courses as well.
As an opinion on selecting schools, consider giving additional attention to colleges with relatively dark and natural night skies and accessible observatories, such as, to varying degrees, Williams, Haverford , Hamilton , Amherst, Wesleyan and Vassar.