You sound much like me when I in high school. I initially considered small schools like Swat and Haverford but decided against them for several reasons, particularly because the opinions from current students on the LGB dating scenes on campus were almost uniformly negative. Ultimately I settled on a school somewhat smaller than Northwestern and similar to it in many respects. I wound up loving it, and it challenged and shaped me in ways that I never expected. College sometimes took me out of my comfort zone, which wasn’t at all a bad thing.
I think there’s no question that Carleton may be a great fit for you. On the other hand, I wouldn’t necessarily assume that being an “introverted, slightly quirky intellectual type” means you can’t be happy at Northwestern either.
There is an unfortunate tendency on CC to reduce each college to a caricature, greatly magnifying small differences between the top universities and liberal arts colleges. You should do your homework carefully to avoid falling into this. In particular, you need to visit each school if you haven’t yet, including an overnight stay if possible.