I made that same choice. It's a great position to be in.
For me, it was the opportunities--the Institute of Politics, the Crimson--and, of course, the city environment. Harvard has Cambridge, a yuppie shopping district, and Boston, a cultural gem of a city, all linked with public transportation (the T). Yale has... well... Yale. With the exception of a few streets of college-catered businesses (which Harvard has as well), New Haven is not the very best city to explore. Tweed-New Haven airport, which I flew into when visiting, is one room. I am not kidding.
These are, of course, two small reasons. I felt more "at home" at Harvard than at Yale, and I guess that surprised me because in no way did I have a less enjoyable time at Yale--I met fantastic professors, heard amazing lectures, took part in engaging discussions and debates with fellow prefrosh.
Don't judge a place on "the people" or their "friendliness." Honestly, people at Yale were extremely friendly. They were
too friendly, like they were trying to hard--I kept wondering when their exams were. And the fellow prefrosh I roomed with at Yale were absolutely horrible. I didn't let that affect my perception, recognizing that Yalies are not all shallow, vapid ditzes; similarly, I would never say all Harvardians are forceful lesbians or retiring Chinese premeds.