I love both of these schools and feel like @EyeVeee has the best advice in terms of focusing on environment.
Haverford, with its Quaker roots, feels more diverse (which is also typical of schools closer to cities.) Colby feels very wholesome but is definitely more remote. Both schools value community.
If you are undecided about major but interested in a few things, look at the course catalogs, the requirements for every student, and the requirements of a few majors and minors you might like. One school might be better for you in that context.
We know lots of happy kids at both.