Honestly it is tough nowadays as the majority of kids will want to “experience the college lifestyle” and get trashed, etc.
However, I would not worry about finding your niche - you will certainly find people with shared interests. (Sports, clubs, hanging out with classmates/hallmates etc). Usually partying doesn’t interfere with a lot of the social groups too much (except frats/sororities - which doesn’t apply to you anyway).
Just find friends who won’t judge you for not drinking/going out frequently - if they do, dump them! And there are always people who don’t go out anyways, albeit probably not a huge amount. If you’re not into partying, don’t let people peer pressure you. People might be surprised at first but they will respect your decision if you stick to your own values and your own game plans.