One daughter was a walk on for junior varsity crew. After less than a month she was asked to step up to varsity heavyweight crew. She lasted one week. Getting up at 5am every day was not what she wanted to do, and was just too tough in combination with classes and homework and studying. Fortunately she considered her class work to be more important.
Which makes this a very good question.
And I also think that Brown is an excellent university, and probably a least a bit less overtly stressful compared to Yale.