Intramural sports are really big at Colorado College, if that's something you're interested in doing.
For activity within the community, I've heard good things about
: Bates, Claremont McKenna, Pitzer, Elon, Lewis & Clark, Tulane, Berkeley, Colorado College, Reed, Hampshire, etc.