@squid5 Lafayette is quite generous in its Excel Scholar program. You can start doing research in the summer after your freshman year, and after that, during academic year or winter break as well. All it takes is that your professor submits a proposal to the academic committee. So yes you can do research not related to your major if you manage to convince a professor to work with you.
Is Lafayette too small? I’m getting a bit of that feeling now because (1) I’m a senior and know most of the schools already, and (2) as I’m applying to grad schools, I have visited some NUs and oh boy they are huge. But I don’t think you will feel like me anytime soon. Classes are hard enough to keep you busy and there are always things to explore (clubs / new gym classes). Also you can cross-register and take classes at any college in the LVAIC group (Cedar Crest College, DeSales University, Lehigh University, Moravian College, and Muhlenberg College).
Now Easton is objectively small and there isn’t much to do in the city. Usually students work hard during weekdays and travel to New York to play hard during weekends 
It takes about 90 minutes to go from Laf to New York (by car or bus), and by New York I mean Port Authority bus terminal, right next to Times Square where all the fancy stuff are. Lafayette’s location is very convenient, especially for international students. If you choose to stay in Easton, there are a farmer market and occasionally food festival, so those also aren’t too bad.
There’s a lot of school spirit around the Lafayette - Lehigh rivalry; I think it’s like the oldest collegiate football rivalry. A year or two ago there was an old car in the middle of campus, which somehow symbolizes Lehigh (?), and a hammer next to it; any student could pick up the hammer and beat the crap out of the car (XX) (more info here: www.lehighvalleylive.com / news/index.ssf/2017/11/howlehighandlafayetteares.html). Now I haven’t seen anything that prominent when it comes to actual D-1 participation but students in general are still interested (some watched Lafayette’s basketball livestream during my computer science lab … argh!). And athletics maintain a respected position on campus.