Harvard used to have a swim test but it was eliminated (I think some time in the 80s). There was a myth that it was a requirement of a donation by Eleanor Elkins Widener because her son Harry had died on the Titanic but that was just an urban legend. Harry did die on the Titanic and she did endow the Widener Library but there was no swimming stipulation.
MIT does require a swim test.