I think if a student has a serious extracurricular interest they pursue, they would want to be in the environment that best allows them to do that. Not in one that involves jumping through logistical hoops. For example, it’s a lot easier for an equestrian to be at a school w stables or a competitive skier to be at a school with a ski team than to try to figure out how to ride or race while at another school.
And yes, if an extracurricular interest is one that is outside the parameters of most boarding schools, than BS may not be the best option.