Yes, it is more than student profile, which is why Trinity is ranked higher than Southwestern in Niche, THE/WSJ, Forbes and college simply, to name a few. They all believe the academics and other factors are higher:
THE/WSJ - #38 Trinity, #89 Southwestern,
Forbes - #110 Trinity, #364 Southwestern
I have spent the past year analysing these 2 schools, and visiting, and I still believe Fiske is out of date re academics.
These rankings, I believe are the reason why OP is questioning Southwestern. He is correct in that Southwestern feels like family. It has a very ‘homey’ feel. It is palpable when you walk on campus. However, there are other factors to consider - such as surrounding area, faculty , ranking, hence his question.
Another thing OP might want to consider, is that Trinity no longer offers a BA in Psychology. You may want to dig into the courses offered a little further, if you were leaning towards a BA vs BS.