With regards to recruits who may have slightly slower times than existing team members, in addition to the potential and known program arguments, some schools, noteably Ivys, have academic targets to hit, so if they take a great swimmer with not so great academics, they can offset this with a good swimmer with fantastic academics.
Also, lets not forget the legacy/development angle. With the pressure for fundraising many college coaches are under, this is a real issue, even if all the rules are being followed.