Limitations and risks of elite admission standards

ivyalumni · December 3, 2005, 4:48pm

I have wanted for awhile to put in a few comments about the ways in which elite university standards for admission create problems.

I have decided to post this to the Harvard board for several reasons. First, Harvard was one of the schools where I was educated. Second, Harvard sets the standards for everyone else.

I. Introduction

On CC and elsewhere there is much discussion of how important very high achievements are. However, less is said about whether these standards of achievement are accurate or useful.

I have been particularly concerned when I see how extremely high standards make students and parents seek near perfection on so many measures of academic achievement. As we all should know, perfectionism can have dramatic negative effects. It leads to countless individuals devoting incredible amounts of time and money to do better and better on tests and grades. Sometimes this means missing out on life and even real learning.

II. Evidence of the overall inadequacy of current objective measures

The measures used in admissions are based on their ability to accurately predict freshman GPA. The main predictors are high school GPA (HSGPA), SAT I, and SAT II.

Unfortunately none of these have proven very reliable. The best study is probably the one done by the UC system. I quote below the results. For more info see <a href=“http://www.fairtest.org/facts/satvalidity.html[/url]”>http://www.fairtest.org/facts/satvalidity.html</a>

After a three-year validity study analyzing the power of the SAT I, SAT II, and high school grades to predict success at the state’s eight public universities, University of California (UC) President Richard Atkinson presented a proposal in February 2001 to drop the SAT I requirement for UC applicants. The results from the UC validity study, which tracked 80,000 students from 1996-1999, highlighted the weak predictive power of the SAT I, with the test accounting for only 12.8% of the variation in FGPA. SAT II’s and HSGPA explained 15.3% and 14.5% of the variation, respectively. After taking SAT II and HSGPA into account, SAT I scores improved the prediction rate by a negligible 0.1% (from 21.0% to 21.1%), making it a virtually worthless additional piece of information. Furthermore, SAT I scores proved to be more susceptible to the influence of the socioeconomic status of an applicant than either the SAT II or HSGPA.

Another way of saying this is that when a test like the SAT predicts 12.8% of freshman grades, 87.2% remains unaccounted for. Even using SAT I, SAT II, and HSGPA and predicting 21% of freshman grades, means 79% is not accounted for.

III. More problems with the SAT

It is well known that the SAT has a standard error of around 30 points. But fewer really think about the implications. On CC, a score of 750 is often seen as a high enough level to apply to the most elite colleges. This certainly applies to Harvard, where a 750 is sometimes seen on CC as almost a minimum.

But what a score of 750 actually means is that a persons true score is probably between 720-780. Since this applies to all three SAT tests, the actual range of error for a person who got 750 on all three parts of the SAT for a total of 2250 actually is scoring somewhere between 2160-2340. For a Harvard applicant, 2160 sounds terrible probably while 2340 sounds much better.

Stop and think of this. Do you think of 780 as WAY higher than 720? If so, you are making a mistake. Then consider how it factors into the total SAT score. There is no real difference between 2160 (720 times three) and 2340 (780 times three). But is that how you think? Is that how the admission officers think?

III. Conclusion

Elite universities such as Harvard have become used to making decisions based on measures that are not very reliable but which are seen to be very important. People can actually think of themselves as an 780 SAT person or a 720 SAT person. The fact that elite universities go along with this legitimizes this sort of error.

I expect one response will be that no better measures are available. Even if this were the case, this is not a good enough answer. First, it doesnt excuse using the measures as if they were more reliable than they actually are (e.g., differentiating candidates on the basis of SAT differences that are not significant since they are within the margin of error). Second, it should lead to efforts to either improve measures or find different ways of making decisions that are more consistent with reality.

What do other CC members think?

Biz17 · December 3, 2005, 4:56pm

wow

(tenrule)

hazmat · December 3, 2005, 4:59pm

**

Out of curiousity…what is your solution to the legacy preference? The acceptance of legacies at a greater rate than regular candidates and sometimes w/ lesser credentials?

Lemonjello · December 3, 2005, 5:45pm

The SAT has always been problematic because of the socioeconomic effects it has. However, if it was eliminated, how would one standardize gpa to account for different high schools of varying academic intensity and strength? Wouldn’t an elite high school that doesn’t rank its students or calculate gpa be significantly disadvantaged despite producing intelligent and qualified students. Would the essays, recs and interviews alone account for this?

Shark_bite · December 3, 2005, 5:50pm

There is a flaw in your logic, A 750 may go as low as a 720, but a 720 will go as low as a 690. this gives you a 2070. When compared to a 2340 this isn’t very good. What I am trying to say is that you can’t assume that the person with the lower score is at their highest and the person with the higher score is at their lowest. When comparing scores, it is possible that the tester is at his lowest score, but is is also entirely possible that he was at his highest score as well. The admissions commitee can not judge a person’s score by saying that “oh, he must have had a bad time and scored low,” when it is entirely possible that the tester was at his highest. The only fair way to judge is to assume that both were in the middle. I do agree that the sat tests one’s test taking skills more then it tests one’s actual capacity to learn, but that is why it is only one piece of the puzzle. Hopefully adcoms (as they are called) judge based on the whole picture and not just one faulty test. I believe that selective colleges usually do this, but often times in a large school, adcoms resort to pure stats. This negatively affects the admissions process as a whole.

(Honestly though, three wrong is a 700 to 730?)

cosar · December 3, 2005, 7:14pm

I don’t know that Harvard (or any other “elite” school) sees nearly as much of a difference between a 2160 and a 2340 as posters on cc tend to see. There’s no doubt that, when they drop below some level, SAT scores make admission to the top schools improbable. But based on listening to Bill Fitzsimmons talk about the admissions process (and this is consistent with the comments made by an unnamed admissions officer at a top school reported in a long thread over in the Parents Forum), I think there’s a level at which the SAT scores are “good enough” and other factors take priority in the admissions decision-making.

Harvard2011 · December 3, 2005, 7:18pm

“producing intelligent and qualified students.”

Schools such as Andover (or whatever it’s called) produce WELL-EDUCATED students… There’s a huge difference between intelligence and knowledge, some people, however, don’t get this and constantly inter-change the two, which drives me nuts and leads to the conclusion that one can “gain” intelligence, which is false.

hazmat · December 3, 2005, 7:21pm

Very well said…there is indeed a difference.

latetodinner · December 3, 2005, 7:48pm

I have been told by psych prof that the 1600 SAT was an intelligence test and a rather good one, especially the verbal section. US News rankings state that at least 25% of Harvard students scored 1580 or above so I guess Harvard likes the SAT. I imagine we have to create a grit and determination test…but then we wouldn’t educate any future poetry profs just captains of industry and their sons or daughters.

runningncircles1 · December 3, 2005, 9:03pm

It was the very first SAT introduced that actually tested intellegence (and was a much better measure than today’s test). It was developed from the “IQ” test which the army gave its recruits in order to put them in proper job fields. Then, it was imported into the college admissions process (mainly from the Dean of Harvard at the time) to test students from different socio-economic barriers and to seek out truly intellegent people, not just base admissions on the legacy system.

The thing that made the very first SAT a good intelligence test (not great, as no great or perfect intelligence test exists) is that one could not prepare for it like today’s. ETS (there was a different name at the time, I believe) was not required to release previous administrations of the test or sample questions. So, when a person took the exam, it truly was standardized: no knowledge of subject matter, no calculators. Only the brain, a pencil, and the little bit of paper given. This standardization is what made it a good measure of intelligence.

Unfortunately, today’s SAT has been bastardized from it’s origins. It is not truly standardized: people from different socio-economic statuses have different levels of “prep,” which is one thing the original SAT wanted to get away from; calculators are used though not everyone can afford them and many have different functions; and some with connections to psycologists and the likes can be diagnosed with a “learning disability” to get more time. Because of this, the current SAT cannot be seen as a good intelligence indicator (at least in my eyes).

coureur · December 3, 2005, 11:41pm

I think you are long on criticisms but short on solutions. What are your reform proposals? What are the true “objective measures?” What does a perfect, or at least significantly improved, college admissions system look like?

hazmat · December 3, 2005, 11:45pm

I’ve said it before but here goes: Scratch Off Tickets

ivyalumni · December 4, 2005, 12:19am

I’m not the genious with the answers; there are many better qualified than me working on this at this moment.

But for a starter, if a system is abritrary then we should be honest about the fact.

Adcoms proclaim appeasing comments about what they do, but too much is unclear and at least some of what we actually see doesn’t look so good.

Okay, the vague comments. “We draw on many factors and make a carefully considered judgement.”

Evidence from informal sources, expert sources, and media sources (the rating system of World News and World Report), suggests excessive emphasis on the near perfect levels of SAT, GPA (everyone is above a 4.0), “rigorous curriculum” leading up to long lists of APs. Last but not least the inflating expectations of “extra curriculars”.

I feel confident that close to half the most talented people I knew at college would not have made the cut by today’s standards. I have no idea if I would have.

As an alumni who graduated more than a decade past, I feel concerned with the increasingly perfectionistic standards (everything in the top top percent) that seem to be justifying selection.

I read down the lists of qualications submitted by applicants on CC-- of numerous major ECs, National and International Honors, perfect APs, stunning SATI and SATII scores. Is this what kids should be going through? And is this really helping them to learn better in High School and to offer more as College students and citizens?

I could accept it better if the standards actually were meangingful. But as my original post suggested, far too many of the measures are of very limited use. All the “objective measures combined” give us only about 20% more accuracy than knowing nothing in predicting freshman GPA. And they do even more poorly in predicting almost everything else (even eventual college GPA or graduation rates). What about that other 80% plus of factors? What is happening to it?

I see a generation of high school students who have been taught that competing comes first (partially regardless of the value of what is being competed for). Students who come to see winning (or at least making it into very elite circles) as the goal of life. Not everyone would say that’s the way to go. Getting into Harvard is a really big accomplishment on this scale.

For those of you who were alumni who got into an elite college like Harvard at another time, how would you feel about having to run today’s gauntlet to get in to today’s elite schools? Would you have made it? How would it have changed the feeling of being at Harvard if you did get in?

So my positive suggestion is this. Harvard and other elite schools have to acknowledge how things have gotten out of control. They should try to provide a model of selection that does not reinforce obsessive and extreme efforts to get into college beginning in seventh grade. They should realize that it is underming education and, through the resulting emphasis on test prep programs, going against efforts to diversify the student body.

I’ll let others provide more concrete suggestions if they want to.

tokenadult · December 4, 2005, 12:47am

I don’t think you’ve even proved your premise here that Harvard mostly admits on the basis of SAT scores.

infectedmushroom · December 4, 2005, 12:48am

I like that theory. It may have something to do with my less than perfect credentials (I cannot compare with most Harvard CCers), but the obsession over the quantifiable is out of hand. I know students who are far higher than myself in class rank because they do exactly what they need to get an A on that test and nothing more. What happened to a genuine thirst for knowledge? Why did I feel pressured to take AP Chemistry, Physics and Calculus when I have no interest in or aptitude for math and science? Students do need to take a step back and evaluate whether they are really learning. I wish I had taken the time to select classes for which I’d have a passion and desire to retain knowledge. Instead I spent four years taking all the “hard classes” and all I have to show for it is a bunch of disjointed equations floating around in the recesses of my mind. It lies on Harvard’s shoulders to be the catalyst for change in the admissions arena.

cosar · December 4, 2005, 1:38am

Harvard is actually quite concerned about the ever-increasing pressures faced by high school students today. (My guess is that other schools are similarly concerned, but Harvard is the school I’m most familiar with.) Bill Fitzsimmons, Harvard’s dean of admissions, has written a very thoughtful piece on these issues entitled “Time Out or Burn Out for the Next Generation,” which is posted on the Harvard admissions website. Here’s the link:

<a href=“http://www.admissions.college.harvard.edu/prospective/applying/time_off/time_out.html[/url]”>http://www.admissions.college.harvard.edu/prospective/applying/time_off/time_out.html</a>

Harvard is not asking for perfect SAT’s or GPA’s - rather, students and parents are simply assuming that’s what’s required for admission. Harvard is looking for highly motivated students, with a passion for learning and a record of accomplishment at a very high level. But this can be evidenced in any number of ways, and perfect SAT’s and GPA’s are neither necessary nor sufficient conditions for admission.

I was admitted to Harvard some time ago, coming from an “alternative” high school with no grades. Having no grades was liberating - I worked hard and did a lot of neat stuff, but I did it because I wanted to. Harvard was incredibly supportive of and enthusiastic about my educational background (unlike some other colleges that basically told me they wouldn’t consider me without grades).

My son was admitted to Harvard last year. There is no doubt that his high school experience was more pressured and stressful than mine, but he did not spend his high school years plotting a course that was primarily directed towards getting into college. While much of his curriculum consisted of required courses, when he had choices, he chose to pursue what interested him. Including not taking math his senior year so he could focus more on his passions, writing and music. He also had summers to himself - no structured programs - and used them as he saw fit, including pursuing his writing and music, but also sports and video games. He ended up with some substantial accomplishments, but these were just byproducts of pursuing his interests, not part of a calculated effort to “look good” for colleges. His approach to summers was remarkably similar to the approach advocated in Fitzsimmons’ piece (though that’s not why he did it that way - we didn’t discover the Fitzsimmons piece until his senior year of high school):

“Bring summer back. Summer need not be totally consumed by highly structured programs, such as summer schools, travel programs, or athletic camps. While such activities can be wonderful in many ways, they can also add to stress by assembling “super peers” who set nearly impossible standards. Activities in which one can develop at one’s own pace can be much more pleasant and helpful. An old-fashioned summer job that provides a contrast to the school year or allows students to meet others of differing backgrounds, ages, and life experiences is often invaluable in providing psychological downtime and a window on future possibilities. Students need ample free time to reflect, to recreate (i.e., to “re-create” themselves without the driving pressure to achieve as an influence), and to gather strength for the school year ahead.”

I agree that the whole college-mania has gotten out of control. But I have a hard time placing the blame for that at the feet of colleges like Harvard.

system · December 4, 2005, 1:38am

The SAT does not test intelligence, and I don’t think it ever did. Honestly, I can’t understand the notion that the SAT has somehow become a less “pure” aptitude test over the last few decades. What did ETS eliminate? Analogies? Antonyms? These are word games, usually associated with memorized vocabulary. If anything, axing them made a better “aptitude” test.

But that’s the problem: the SAT isn’t really an “aptitude” test. It does have a correlation with aptitudeprobably a fairly decent correlation. I’ve never known an academically brilliant person with a low score, and I’ve never known a positively stupid person with a high score. At the same time, however, the SAT suffers from a number of flaws; some are due to inevitable problems with a multiple-choice format, but many are due to a poorly designed test.

Take math scores at the high end. I’ve known plenty of people who received 780-800, yet can neither solve problems creatively nor comprehend rigorous mathematics. In fact, there are people with 800s who seem to actually be bad at math, or at least lacking any intuitive understanding of it.

The problem, of course, is that the math SAT consists of easy questions with a beastly curve. It becomes a competition to see who can make the fewest mistakeshow can it possibly be construed as an accurate measure of math ability?

The Math IIC seems to sort mathematicians somewhat more capably, which is ironic given its perceived status as an "achievement test"as opposed to the absurd “aptitude” label given the SAT.

The reading section of the SAT usually has a number of poorly designed questions. These lead students to “overthink” the exam. They often use quite sophisticated reasoning to arrive at an incorrect answer, while the College Board wanted simplistic analysis. It doesn’t have to be this way. Intelligent test makers could quite easily write answer choices so that only one would be “correct”. No garbage about “more correct” or "less correct"just one answer choice that could be reasonably inferred from the passage.

I don’t mean to tar every question in the sectionlooking at the College Board’s sample tests, it appears that each exam only has a few real offenders. Any poorly written questions, however, distort the predictions of a test.

Anyway, I’m not against tests. I dislike the SAT because it is a poorly constructed test (and this isn’t just sour grapes; I got a 2400). We could have a far superior testor series of testswith far greater value.

system · December 4, 2005, 1:41am

Wow. This seems to be pretty solid evidence supporting either the SAT’s reform or its replacement by SAT IIs, which appear to be the best available predictors of college grades.

runningncircles1 · December 4, 2005, 2:05am

randomperson, what do you mean that the SAT has not become less pure? It has become less pure because it is no longer standardized. It isn’t the fact that ETS got rid of things from the test that makes it bastardized, it is the fact that the levelled field for which the test was made has been compromised.

Flavian · December 4, 2005, 2:05am

Two points:

One, I think its widely recognize that the SAT is not that good as an assesment test. However, the problem is that if there were no SATs, there would have to be a different standardized test. What are the candidates for an alternative? the ACT - well its basically like the SAT but with different-styled questions. We could have a mensa IQ test, but that doesn’t measure academic motivation, which is usually more important than “intelligence.” We can have something like a universal AP or SATII subject test, but the problem with that is it becomes heavily baised toward prep schools. I’m sure you met many productive and intelligent people from poor schools who would not have done well on AP or SATII exams simply because their high school wasn’t that good.

The thing is, compared to the college admissions of other countries, the U.S. is actually very flexible. In other countries, one test determines your college. I’m sure you can agree that these tests are even more “worthless” than the SAT. At least U.S. colleges take into consideration your ECs.

Secondly, it is not possible to find a factor that fully predicts sucess in college. “Success” is an up and down thing. The most brilliant kid in high school can turn into a slacker and become very unproductive. A street bum can become one of the most brilliant scientists. I’m sure Harvard has faculty members who did not even attend undergraduate college. So in a way, it is unrealistic that you can assume a near perfect predictor.

You might respond: well how come colleges still put so much weight on SAT scores? Well the answer is that there is no hard evidence to say that’s the case. I would argue that SATs are not that important at all when it comes to Elite universities, once you’re past a certain point. So the discrimination is not between 1600s and 1450s, but really, between 1550s and 1100s. In this case, I think its pretty obvious that there is a difference.

Finally, I want to say that Elite colleges aren’t exactly looking for people who will do best in college, but people who will do best after college. In this case, its even more unrealistic to find a suitable test for prediction.

Limitations and risks of elite admission standards

CONNECT WITH US