From wikipedia(although not a trusted source, it is cited):
While it is widely reported that Smith turned down a scholarship to attend the Massachusetts Institute of Technology (MIT), he never applied to MIT, although he was admitted to a "pre-engineering program" there. According to Smith, "My mother, who worked for the School Board of Philadelphia, had a friend who was the admissions officer at MIT. I had pretty high SAT scores and they needed black kids, so I probably could have gotten in. But I had no intention of going to college."