So you’re a sophomore and you’re just learning of the culture at the school now? It worries me that you say, as you hear from upperclassmen - as you’ve been there a year.
So as you know, there’s a party culture at many places. But I believe one can also find their crowd anywhere- but perhaps it’s more difficult at a smaller school.
Are you seeking a bigger school? Your academic opportunities sound awesome. Are there clubs you can join, etc? For some, it’s just hard to make friends - it’s understandable, especially in times of covid. But if you move schools, you go in not knowing anyone.
To me - especially if you’re financially constrained, the most important thing is the aid you get. Yes, you need to be happy and I get it - but you don’t want to end up at a school you cannot afford or that brings financial stress.
It sounds like you’d like it to work there. I’d focus less on what people tell you and more on what you’ve experienced. I would seek out clubs in your interest group.
If you do decide to transfer and I assume go to a larger school, make sure it affords the opportunities you want - research, etc. and of course, check on the finances.
Only you can decide - I wish you luck.