<p>
</p>
<p>Uh, no, not really. The KD study controlled for aptitude as measured by standardized test scores, which is only one dimension of aptitude, and, frankly, a relatively minor one. No-one would dispute that ‘aptitude’ is a far broader characteristic than simply test scores. Every year, Harvard and some other top schools could admit an entire class of students with perfect 2400 test scores. But they don’t. We can debate whether the admissions criterion used by the top schools are appropriate or effective, but there is no dispute that you need far more than merely a high test score to be admitted to those schools. </p>
<p>Furthermore, the authors unfortunately introduce yet another ‘aptitude’ variable by comparing those students who were admitted to HYPSM and matriculated vs. those students who were also admitted yet chose to go elsewhere. That latter choice is itself correlated to (unobservable) features of aptitude or other opportunities. Perhaps those students are unusually self-confident, in that they feel they can succeed without the benefit of a HYPSM education. Perhaps those students are turning down HYPSM for a special program at a lower-tier school such as a BS/MD program or other opportunity available only at that school. {For example, I know one guy who turned down an Ivy for an academically lower-ranked state school that also happens to be one of the most famous football schools in the country, with him being a top football recruit who wanted to take his best shot at the NFL. Even if he doesn’t make it to the pros, he nevertheless enjoyed a 4-year full athletic scholarship and the experience of playing for a football powerhouse, including several prominent bowl games.} The upshot is that those students who are admitted to HYPSM and then deliberately choose to go elsewhere are clearly not comparable to those who are admitted and actually go. </p>
<p>Unfortunately, the only feasible way to truly ‘control’ for the unobserved aptitude variables, short of a randomized experiment, is probably through a regression discontinuity: compare those HYPSM students who were barely admitted vs. those students who barely missed admission. For example, you could compare those students just before and just after the wait-list cutoff: if Harvard matriculated 50 students from the waitlist, then students 41-50 from that waitlist are highly comparable to students 51-60 in terms of aptitude. Then you could truly measure the discontinuous treatment effect of going to Harvard. Good luck trying to obtain that data, however.</p>