Can you clarify why it is interesting that the authors are independent professors as opposed to admissions personnel? An author’s independence is often a prized attribute.
1 Like
I’m sorry, but the fact that colleges want to use the SATs for admission in no way or form proves that the tests aren’t crude. Colleges use crude and biased factors for admissions all the time, and have a long history of doing so.
Part of running a large entity like a university is spinning facts the fit your narrative. So outright lies and deception are rare, since these can turn around and bite you, but overstating things, selective presentation of facts, etc are part and parcel of college announcements. Colleges want to use SAT scores for their own reasons, but, because SAT scores are associated with wealth, they possibly feel the need to overstate the predictive abilities of SAT scores to counter the narrative that they are using SATs in order to increase the number of wealthy students who are accepted.
I don’t “want it to be true”, I know that is true. The College Board itself agree with me.
According to their own analysis, and I will quote them here:
Something that explains only 15% of the variance in the dependant variable is pretty crude, by any standard. You can claim that an explanatory variable that explains 15% of the variance in the dependant variable is “an accurate predictor” of the dependent variable, but, as you wrote:
Nope.
15% is a significant, material % in regression analysis.
Sigh.
With a sample size of tens of thousands, even the slightest effect will be “significant”. An r^2 of 0.02 will be significant.
Of course, that doesn’t matter. I went back to read the paper, and no, the SATs are not explaining 15% of the variance in the model. Adding SAT scores actually increases the correlation by 15%, from 0.53 to 0.61. If “the correlation” is r square, tha means that adding SATs increases the amount of explained variance by 8%, which it’s really low.
However, r squared is generally called “The Coefficient of Determination”. On the other hand, r is called “the Coefficient of Correlation”. So they are more likely presenting the r of the correlations, and that means that adding the SAT scores increases the amount of explained variance from 0.281 to 0.372, so adding the SAT scores add another 9% of explained variance.
However, neither r nor r squared will tells you how accurate the model is. It just tells you how much of the variance is explained by the variables in the model. Also, to make their claim about how much SAT adds, they probably should have calculated its partial r squared value.
I would also love to know how they “corrected” the correlations. There are many legitimate reasons to do so, and many ways to do so. However, most adjustments reduce Correlation coefficients, so I would really like to know what they did.
So, in fact, they have actually said very little about how accurate SATs are at predicting first year SAT scores, nor do they really provide any accurate information on how much better the model performs when adding SAT scores.
What is certain is that it isn’t a lot.
PS. Comparing their results to those presented by the UC people, the are, indeed, providing the r values, not the r squared values.
Not pointing at you or anyone else. But all this jargon makes me recall the phrase “lies, damn lies and statistics “. Mark Twain attributed it to Disraeli.
The SAT was originally a tool which Harvard, Yale etc used to get beyond the prep school old boys’ circle. It’s also now part of the brand of these schools, i.e. high SAT scores = elite meritocracy. The public doesn’t care about the statistics on correlation with first year grades or the like. I suspect the admission officers also shrug at statistical studies in this area . The admissions people view it as useful in this era of grade inflation. As one example, an admissions officer at an Ivy told me a SAT score is an important check on foreign applicants whose grades might be suspicious or difficult to evaluate. SAT is both part of the brand and a useful tool for assessing applicants. Contrary to many predictions that schools would stay TO after the affirmative action Supreme Court decision in order to obscure potential discrimination the elites are bringing back test required. SAT is here to stay
2 Likes
The numbers that jump out at me:
While the likelihood for students to score high enough to meet a minimum threshold for highly selective colleges (I think in this thread, generally assumed to be a 1400) or to score high enough for the score to actually be helpful for admission to highly selective colleges (say, 1500) is clearly strongly correlated with income, the actual percentages of students who actually manage this is still incredibly low!
The narrative often appears to be framed as if high income families can “buy” the high scores for their students, not by illegal means but by moving into neighbourhoods with good schools, paying private schools, offering enrichment, paying for tutoring…but still among the top 98th percentile in income, it’s still only just over 10 percent who manage a 1400 or above? 99th percentile, everything that money can buy…but not, apparently, success on the SAT, its still only 5 percent who manage a 1500?
And look at how many, even with all the advantages money can buy, don’t even manage a 1200?
Clearly, those scores mean something, even across income groups, but even more so within income groups! Think of how many students will apply to highly selectives from the 90th percentile and above in income, how many of them will have 4.0 GPAs from comparatively well resourced high schools with comparable rigour? (And, of course, carefully curated ECs, volunteering experiences, edited essays, but SATs are about evaluating academic potential, and whatever role athletics and character and grit are desired to play or not, the academic evaluation has to be done).
That’s where 1300s, 1400s, 1500 become meaningful, putting the other measures in context.
And what do you do with the FGLI applicants with 4.0 GPAs but without access to AP or DE classes and either no idea how to gauge the school’s rigour or a fairly good idea of the lack of it…how do you put those GPAs in context? You can’t. You need a minimum SAT to know that in admitting the student, you are doing the right thing.
Because helps them admit the ones who have a good chance to hack it, who may struggle to adjust, but who will adjust.
(That’s the ones Dartmouth and Yale are talking about.)
And it also (that’s the ones they’re not talking about) helps them reject the ones who they expect to struggle badly and who may be demoralised and drop out, or somehow make it out but not with the major they wanted and the grades they needed to be successful.
(Because admitting them isn’t good for them either.)
When everyone had to submit SAT scores, admitted students to highly selective colleges all had SATs within a fairly close range - of course it doesn’t explain variance well if there is so little variance, and the same goes for GPAs - all high.
Where is the variance, always has been? High schools! Rigour! And that’s why “name of high school” and “GPA adjusted for rigour” is so predictive!
With test optional, suddenly the SAT spread opened. Less self selection by SAT score among applicants, less selection by AOs. Less context for schools because fewer students take it at all.
And thus, suddenly SATs became predictive again. Because test optional wasn’t a niche phenomenon any more.
7 Likes
Not really, or, should I say, in an extremely limited manner. It was used to increase the number of wealthy White Protestants who were not attending one of the regular feeder schools for the Ivies. It was, however, also being added as another barrier to Jews and other minorities.
The SATs were based on the army IQ tests, which was also the test that was used on Immigrants to test “IQ” of people of different origin. We all know that Jewish immigrants did not perform well on those, nor did any of the “undesirables” that the Ivies were trying to keep out of their Hallowed Halls.
Of course, like today, the proponents of those old SATs were adamant that the SATs were Fair, Impartial, Objective, Accurate, and, most of all SCIENTIFIC. Any attempt at pointing out how biased they were was attacked as being “Anti-Science”. It’s not surprising, since, for the first time, racists, misogynists, and antisemites could claim that they had absolute, unambiguous, and scientific proof that they, the Wealthy, White Anglo-Saxon and Nordic men, were Superior To All Others.
So no, SAT tests were never added to make admissions more meritocratic, they were added to expand the pool of Wealthy White Protestants to replace the Jews and others whose numbers were now being slashed or eliminated entirely.
There is really no way that a test adopted by the same people who invented discriminatory admissions standards was supposed to be inclusive.
I agree, especially about “part of the brand”:
However, I think that the smaller “elite” LACs will likely stay TO, since they have always put less emphasis on SATs to begin with, and “We Have The Students With The Highest SAT Scores” has not been an important part of their branding.
Most publics, except for CA colleges, will also likely not be TO, since SATs are pretty convenient for colleges which don’t have the resources to look through every single application in detail. Using the SATs they can cheaply and quickly reject half of the applicants that they want to reject. Even with all its flaws, the SAT analysis about the graduation rates of applicants with SAT scores below the roughly 40th percentile are probably accurate enough to justify removing these from consideration. This is especially true for public universities that are performing a trade off between efficient use of public funding to support students on one hand, and not contributing to increased economic stratification, on the other.
2 Likes
I don’t think the SAT was used as such to keep out Jews. The SAT was used in the 30s for scholarship students at Harvard. After WW2 started, it became more widely used especially at the Ivies which were developing into elite meritocracies for Protestant Americans beyond the typical prep schools (as you noted.) The Jews did quite well on the SAT so it was not a barrier to them. They were kept out of the Ivies by outright quotas or the use of “personal characteristics” (sound familiar?). That started fading away by the 1960s–when I went to Yale in the late 60s., it was 30% Jewish.
2 Likes
Those were the facts, but those facts were only obvious after a few decades of these tests being administered. The original “IQ” tests, the one that was administered for immigrants, the one that those early SAT tests were based on? Jews did not do well on that at all.
The person Henry Chauncey and Wilbur Bender, the two men tasked with the President of Harvard contacted to help them to figure out a “test” for scholarships and then admissions, the person who advised Harvard on the adaptation of the SATs, was none other than Carl Brigham. To anybody who is not familiar, Carl Brigham was Princeton Professor and a leading eugenicist. He was the person who wrote A Study of American Intelligence, the book which was responsible for setting the standards for all “IQ”-based racism of the past 100 years.
So it is 100% certain that Henry Chauncey and Wilbur Bender, as well as James Conant, the president of Harvard, were certain that the SAT tests would favor members of the “Nordic Race”, because the man who created the tests and who advised them on administering and scoring them believed this to be true, and it was a major premise of his book.
There is absolutely no way that the Harvard people believed that Jews would perform well on the SATs, and every reason to believe that they didn’t.
The fact that SATs did not, at the end, prove to be the barrier to Jewish applicants that Chauncey, Bender, and Conant, and even more so, Carl Brigham, hoped it would be does not change the fact that they thought that it would be a barrier, and it was put in place to be such a barrier, while still allowing more middle class “Nordic Europeans” to win scholarships and, the next year, to be admitted.
But the SAT,in fact, wasn’t a barrier to Jews, right? The Harvard scholarship program adopted the SAT in the 1930s. The history you cite deals with the time of its creation pre WW2. By the time the SAT was widely adopted during the post WW 2 era, that fact that Jews were not inferior on the SAT was becoming obvious. So any purported earlier assumptions of “Nordic” superiority to Jews on the SAT had been confounded and could not be a motivation for widespread adoption and use of the SAT.
The same historical irony applies to other groups like Asian Americans who were viewed as inferior on the IQ test. The Supreme Court affirmative action case used their SAT scores to strike at the use of “personal characteristics “ to keep down the number of Asian Americans at Harvard.
The history lesson I draw is that the SAT has backfired on Harvard.
3 Likes
More than anything posted in this entire thread, this statement works against your zeal for proving that the SAT is an unnecessary barrier. Factor this in with the truth that “holistic admissions” were invented as a ‘remedy’ to standardized tests, in order to keep out Jews and other ‘undesirables’ who test well, and the answer is clear. We should do everything we can to move away from holistic admissions, and anything that factors in personal opinion, and shift importance in admissions to the merits of the individual, regardless of race, religion or any other demographic category.
3 Likes
The problem is that, as with the SATs, “non-holistic” admissions standards are usually implicitly biased, but create the dangerous illusion of being nonbiased.
But the big reason that “holistic” admissions are here to stay is, at least in the smaller private universities, is that they are not looking to admit the “most worthy” one way or another. They are not looking to find the ones who are “only good” at one type of academic, or just good at academics, etc. That part of their mission has not changed, even as the type of students that they are looking for has changed. They want graduates who will be powerful, influential, and/or famous for positive reasons.
These colleges see their students as “investments”, in that sense. Even somebody like me, with little experience in financials, can say that it is always best to have diverse investments. Using one set of criteria to choose all students will not be their best "investment " strategy. There are all sorts of characteristics that determine success in life, and relying entirely on SATs or on SATs + GPAs means that the other characteristics are being ignored.
As for “fairness”. Yes, many of the “holistic” factors are also biased, or even highly biased. However, they are usually relegated to the “considered” category, whereas Test Scores, were, uniformly, given the status of “highly important”. So their impact on admissions, and thus the effects of their implicit bias, is much more profound, even if that bias is not as extreme as that provided by “accomplishments” or “special talents”.
However, the great majority of colleges, i.e., large and medium public universities, are not selecting students based on the illusion that an SAT scores of 1510 demonstrates a higher degree of mastery than an SAT score of 1410. Most use SATs as the crude tool that they are, and differentiate between large scale. One just has to look at the SAT ranges of their applicants and admitted students.
Everything, and I mean everything that is fact-based, that has been posted in this thread suggests otherwise.
Are you talking solely about academic merits here?
Many factors besides SATs are biased, not just “holistic” ones. Things like GPA, rigor, and essays are all biased and often categorized as Very Important, not Considered.
Also, Harvard lists virtually every factor as Considered, and Yale lists a number of “holistic” attributes as Very Important, including ECs, Talent/ability, and Character / personal qualities. So I am not certain how consistent the assigning of values is across schools, or how much they actually tell us about what a school thinks is more important.
1 Like
End of the day, for the vast majority of college applicants, no one cares about what the IVy’s or “selective schools” do and how they rationalize the gaming of their admissions process. The schools that don’t play the SAT game seem to have an approach that works for them. And for the posters that seem to want to refute that the SAT pays lobbyists at the state and federal level, it is well documented. And then there is this too.https://thehill.com/homenews/education/4466515-college-board-pay-750k-selling-new-york-students-sat-data/
This article is about state officials fining the College Board for selling student data, so I guess the lobbying efforts weren’t very successful.
In any event, state and federal officials do not determine the admission policies of private colleges, and thus the lobbying efforts that the College Board aims at government officials has not been denied or even discussed much in this thread.
My point is that College Board’s motives are about “money” and not the success of students. There are numerous articles about the millions of dollars the “organization” has in off-shore accounts. So when school’s moved to test optional and planning to do so for a foreseeable future, they poked the bear. So now, we are subjected to these studies that take “incomplete data” to try to argue that the SAT is needed.
If the SAT truly cared about students, they would offer “hours” of free preparation to all students ,including tutoring. They have the money and surplus to do a lot more like a real not for profit would. Instead it is documented that over 20 employees of the College Board earn $300,000 or more pre-pandemic.
This is all just a big, fat red herring.
The “preparation” was the previous 12 years of compulsory education. Still blows my mind people think that the SAT and ACT blindsides kids with some novel area of human knowledge they’ve never encountered before. “2+2=4?? What is this witchcraft??”
9 Likes
Great point, made in a humorous way. Well played. Bravo!
2 Likes