As the threads on this topic show, you can't really control what is going to strike them on a visit -- it might be any old trivial thing that makes them say, uh uh, not this one.
So even though the weather should not make a difference and gives a more realistic sense of what the place is like for most of the school year, I can't say it won't make a difference to your kids.
I think I have said this before, but if it's a school they are excited about, it may not matter. If it's a school that you are hoping to "sell" to them, then try to go on a beautiful day in May
!