Exactly. No one ever knows, sees or cares if the glee club stinks.
But folks do notice if the football team loses 50-0. Even in the Ivy and NESCAC leagues which are no revenue/light attendance/small time as far as college sports go.
Also, those schools wouldn’t bend the rules to award athletic accomplishment in a sport the school doesn’t sponsor. Being a great cricket player gets treated the same as a great violinist; being a great quarterback is a completely different thing.