Note that some graduate programs, particularly PhD programs, are funded by the institution you attend.
As examples, here you can read some brief, subjective comments on several of them: Struggling with D21's List. ED & ED2: Amherst, Hamilton, Wellesley, Vassar - #7 by merc81.
Based on some of your comments, look into Haverford, Swarthmore, Vassar, Reed and Carleton.