My son was Lville class of 2018. He was in a similar situation (choosing between Andover and Lawrenceville), and ended up at Lville.
The current headmaster has quite tone deaf, as pointed out above. But even when my son was there, the squash program was undermined for no particular good reason. The headmaster didn’t meet with parents and showed no sympathy for our concerns.
But I don’t think this is unique to Lawrenceville. Each boarding school will have its own set of issues. Some good, some bad. While Lawrenceville helped my son grow, he probably would have been fine if he stayed at the local public school. The house system at Lville does seem to engender some school spirit.
Andover is a great school but note that it is considerably larger than Lville. Almost like a small college. It also seemed to have more of a sink or swim attitude. But I also found the environment warm and welcoming.
I surely wouldn’t choose for prestige’s sake alone. That is somewhat meaningless when it comes to college admissions. In fact many colleges are looking away from the traditional boarding school feeders.