Definitely not too early!! Absolutely definitely not.
We live in a college town (big state flagship) and our local university has a program called “First Look” especially set up for middle school students so they can get an idea of what college is like. My D22’s middle school class did that program and it was really helpful for her just to get an idea of what everything was about (dorms, cafeteria, etc).
We started visiting colleges (other than our local one) around middle school for my younger kid (went on some official organized tours with older sibling) and we walked around a lot of campuses, too.
Other things I would encourage are the Splash! programs at various colleges across the country and a summer camp program for high schoolers on a college campus. I think the camp is super helpful if you can swing it. We did both a free day camp at our local university and a paid weeklong overnight camp in one of her areas of interest. The overnight camp was incredibly helpful, but the free day camp was also good (and free!).