On my freshman hall, it seemed every other guy was premed at the start. Although 5 or so did end up graduating from medical school, a fair number changed their minds pretty quickly. I think that is pretty standard. For that reason, I would recommend not basing your choice on premed.
The only other advice I can give you is once you decide, don’t look back. Much is made of college admissions, but the school itself doesn’t determine your future. You’ll need to make your own way.