We started late. We visited a well known local (1hr away) school right before the summer before senior year. We knew we couldnt afford visiting very far schools at such a time crunch, so we made a plan. We visited a very large state school, and a smaller private school that were within 3 hours of us during the summer. That way my kid would have some sense of what he would prefer. Any school that was super far, we used virtual tours. During the beginning of 12th grade, my child was invited to visit one college for the weekend and he traveled by plane.
This worked out well, however for my younger kids I would probably start sophomore year so that I could plan visits with vacations.