We’re also from CA and in the same boat of Haverford being one of our kid’s final choices and her agonizing over the decision. One week to go, and a decision will be made by then. So we’re almost at the finish line!
Clearly, you’re more likely to get a good reading on vibe if you visit when students are there. But even then, if you hit a week of exams, the period when everyone is getting to leave on vacation, the first week or two when new freshmen arrive, etc., it might not be representative either. Or if the weather is bad or…
We liked our summer visits - of which there were many as DS played both spring and fall sports and the school-year calendar was chock-a-block. They were more relaxed, and the students and faculty who were on campus seemed to have quite a bit of time to chat. Our CC advised NOT to ED anywhere that we hadn’t visited in session, but for many of the schools, the summer tours/info sessions were very helpful. And a few of them were on the "admitted student " circuit we did in the spring of senior year.
In your case, as you’re close and could get back if you wanted/needed, it’s probably worth it.