The Misguided War on the SAT

What? These are…not the same things at all.

My two cents:

The extent to which processing speed should, or should not, be considered a part of the measurement of g (general intelligence) is discussed in neuropsych literature and is controversial, particularly among experts in twice-exceptionality (gifted with learning disabilities).

SAT does not purport to measure general intelligence. It purports to measure academic skills. I suspect that SAT-taking speed is perhaps a subset or one type of processing speed. Many things - from anxiety to auditory processing to aspects of vision - can impact SAT-taking speed that may or may not be relevant in various professions. These things tend to sort themselves out later in profession choice such that the SAT does not need to be a gatekeeper.

I’ll add that ACT is far more known for being a time crunch than SAT. For those without extended time accommodations, as a practical matter, the SAT is often a better choice.

(To be clear, it seems to me that the SAT has utility for college admissions. I just don’t think that tight timing is particularly relevant to the academic skills it is attempting to measure.)

2 Likes

A good restaurant has good food. Ambiance, service, and promptness are nice to have but not as important.

2 Likes

The SAT having a timing component makes it an exception to what rule? Every test in every high school? Every other US standardized test? Every other national examination given in the world? But the SAT should have no time limits?

And anything is “controversial” especially "as applied to a very limited population like the exceptionally talented with learning disabilities. Whether the Earth is flat is “controversial”. Whether species evolve largely as a result of the process of natural selection is “controversial”. Nobody is changing the maps though, and the vast preponderance of science books chug along with Charlie Darwin. Likewise, “the controversy” in other tests, cognitive, academic or otherwise have not prevented the overwhelming majority of actual evaluations to require a timed component.

Rather than search the universe for occupations where long periods of meditation are of paramount importance or examples of the slowest student in the class who is the deepest thinker, is it such an unacceptable notion that for the occupations that the majority of students at the most selective universities are targeting, the ability to perform accurately and quickly is important?

4 Likes

It is not so much that thinking fast is unimportant, it is that it is overvalued. In all the professions you list speed can be important, but so can careful and thoughtful contemplation, as well as a host of other skills. The SAT heavily weights the importance of one, but not the others. It loads the deck with regard to one particular skill among a wide spectrum of relevant skills.

Imagine choosing a surgeon based on how fast they complete their operations rather than their results.

I see no basis for saying SAT values speed over other attributes of academic potential. And most people can finish the test in the allowed timeframe anyway. This conversation started with concerns that accommodating certain disabilities was being abused by privileged individuals. A solution was to just extend the time for everyone. That proposed solution at least creates an even playing field. Or just stop making accommodations.

2 Likes

The solution does create an even playing field as it doesn’t just address the gaming of the system but also a lack of accommodation for those who may have an undiagnosed LD, many of whom are likely to be from lower socioeconomic circles where there isn’t even knowledge about testing accommodations.

3 Likes

Almost 2,000,000 higher education faculty and another 2,000,000 engineers would beg to contest this claim, not to mention songwriters, authors, lab techs, etc…

First, a LOT of planning goes into each surgery before the surgery, and how much training goes into those supposed “quick decisions”. It is mostly training that allows them to do this.

However, more importantly, medical practice is NOT intellectual work, despite the rumors and the stories. It is technical - a physician’s job has more in common with that of a car mechanic or a computer repair person than with that of a scientist. It’s an order of magnitude more complex, but it’s not qualitatively different.

If you have ever had a really good car mechanic, you’d know what magicians they are.

Like medicine, the practice of law isn’t an intellectual pursuit , but it’s all about having really good associative memory. Of the total 1.3 million lawyers in the USA, only a small percent are actually “trial lawyers” or litigators. I’ve found numbers of less than 10,000. The rest of the attorneys can take their time, bringing us to:

I’ve worked in a job with “billable hours”, and am doing so now now. No, you’re not working at the speeds that SAT require - it’s the speeds of “take-home exams”, or slower. I don’t have to produce accurate results in two hours, in a room with no access to records or literature

Not what I would consider an “intellectual profession”.

As for:

I think that I know a lot about “research scientists”, seeing that it is what I did for a few decades. I can tell you, unequivocally, that there is nothing in common between anything that any research scientists does and the skills required for the SATs or ACTs. Nothing.

I cannot see the slightest similarity between what you are describing and the skills required for high scores on the SATs.

Most importantly of all - not a single one of what people do in these careers works like the SATs.

The SATs all work in a very specific manner - they start with an answer that is know, and then craft a question around that answer. Not only is the answer already known, but there is only one single correct answer which is definite and unambiguous.

Every single intellectual endeavour, bar none, works in a completely different way. You start by formulating the question, or, if the question exists, the answer is unknown, and there are likely multiple answers.

  • A physician would fail in they studied medicine like you study for the SATs, because people differ from one another.
  • If a lawyer practiced law the way that they would answer the SATs, that lawyer would fail, because the entire point of having a lawyer is that there are many possible ways to interpret any given law.
  • If a research scientists tried to do research like they would do the SATs they wouldn’t be able to do a single thing.
  • If a musician would try to play music like the SATs, they would be sitting on stage, waiting for questions.

Etc, etc, etc.

Finally - the biggest reason that the SATs are not useful for testing for intellectual capabilities is that they are 100% incapable of measuring the most important factor in the success of anybody in an intellectual pursuit - creativity.

2 Likes

Those guys at Jiffy Lube should demand a raise.

3 Likes

All of the examples you gave are for professional or post graduate pursuits. You are mixing peanuts and apples. Of course the SAT/ACT doesn’t test for those specific jobs. Otherwise why do the MCAT,LSAT, GRE,ABET etc. exist? Or why are there auditions for music or drama school? But all those professional pursuits assume some core capabilities to do college work and the SAT/ACT correlates to those capabilities to some degree. You really must be able to read complex concepts in a short amount of time for rigorous college work. College is necessary but not sufficient for those professions.

6 Likes

To be fair, neither do 90% of school assessments. I don’t see people railing against them.

7 Likes

But is anyone really railing against them? I think it is more of a pushback against the idea that they are determinative - as if a student with a sub 1500 SAT isn’t capable of doing well in school or in one of the professions that seem to be so prized on CC. One of my kids got a mediocre, lopsided SAT score (700 ish EBRW/500ish Math) and has excelled in college - first at UMass and now at Chapel Hill (where he tells me he is often the first to finish the exams - seeing as “speed” seems to be a critical measure according to the denizens of CC). My second son killed it on the SAT (1580) and is also doing well - at UVA. If the SATs were as predictive as many here seem to indicate, you’d figure kiddo #1 would be struggling in some way, and he is not (to say the least).

2 Likes

Because school assessments are not supposed to be there in order to test for intellectual capabilities. They are there to test whether the student has mastery of the material.

SATs are the exact opposite.

First, the kids have years to read and minutes to write. The SATs are supposedly assessing the material that the students have studied throughout high school and middle school.
Second, in the part of the SATs in which students are supposed to answer questions on a short reading, the students have a far shorter time to read the material on which they are being questioned than they would get in college.
Third, in that section in which they are being asked about text that they get, the testees have a multiple choice format, which means that they are (as I mentioned), going backwards. They look at the four or five possible answers and then see which one of them is the best match to the text. In colleges classes, they will have to look at the text and then try and formulate their own idea of what the text means.

MCATs and LSATs are specifically to test for the abilities of these students to memorize large chunks of information, and recall them on demand. They test data retrieval, not reseasoning, logic, or creativity. For law and medicine, data retrieval is an important part of the job. However, the main reason that they test specifically for this is because it is the only skill and talent that can be tested using a multiple choice test.

Plenty of studies have demonstrated that GRE scores are not correlated with the success of students in their doctorates, which is why many of the very top programs are dropping the requirement. In any case, all that it tested was whether the student had the required vocabulary in the field. At no point do the GREs actually test whether the student has the ability to do research.

ABET is an accreditation assessment for a program, not an individual. Engineers are people which an engineering undergraduate degree from an ABET-accredited program. SO they have to have passing grades on the required courses, not score a minimum on some test.

You probably mean the FE and PE exams, which are a lot closer to actual work conditions than the SATs - they have more time and access to their reference materials. They also have about half as essay questions. Finally, it’s pass/fail, not this ridiculous situation in which deep meaning is assigned to minor differences in scores.

The FE and PE tests are also CYA. If a bridge fails or an engine fails because of a design flaw, and it turn out that one of the engineers on the project cannot do basic calculus, the company is in trouble. However, if they are required to pass the PE before being hired, any lack of required engineering skills and education can be blamed on the failure of the test, and therefore of the NCEES.

1 Like

A clarification, the LSAT is reading comp and logic. In contrast, the bar exam is data retrieval.

1 Like

Thank you for the clarification. However, looking at their examples, I would not consider it as testing for logic abilities as much as for the ability to identify the use of logical reasoning or fallacies in somebody else’s arguments. Again, it is what they can test with a standardized multiple question format, and it is a critical skill to have as an attorney.

It seems that, while the LSATs are better at actually testing for required skills, they commit one of the fallacies that the SAT does, and likely for a similar reason. The question is whether you want to test how well applicants do in understanding complex text, or how fast they can do it. Prioritizing the ability to analyze complex arguments is more important, but prioritizing speed is cheaper.

It all comes back to the fallacy that “smart” = “fast”, a fallacy which is deeply embedded in USA culture.

It is so deeply embedded in the culture that people go through extremely convoluted reasoning to prove that speed should be tested as a critical intellectual talent across all intellectual endeavours.

I would like to point out that it took Einstein ten years to formulate E=mc^2. I’d like to see how somebody would test for that ability through using time-limited multiple choice questions.

1 Like

Are you open to the idea that the anecdotal experience you’ve had with your two children may not be a legitimate comparison to the comprehensive reports involving thousands of students that have been done by experts at the top universities that all decided there was value in bringing back requirements for standardized tests?

2 Likes

It all comes back to your rejection of the idea that general measures of intelligence like g or FSIQ should be part of the criteria for college admissions or (seemingly) for jobs, and that those decisions should instead be made on the basis of achievement/knowledge rather than (what most would describe as) potential.

Tests like the SAT and ACT are designed the way they are because their results correlate with those measures of intelligence, and not just your base knowledge. That’s why they have been used for many years with elementary age kids to identify exceptional talent.

It’s not a specifically US cultural thing to value “intelligence”, it is something intrinsic to all societies. It is also something that most seek in their partners and in their leaders. And in a deeply unequal education system, where students may not have had the same opportunities to master specific aspects of knowledge, identifying potential “diamonds in the rough” is particularly important. As MIT indicated when bringing back the SAT.

5 Likes

I don’t think my experience is universal. And there is the chance that had kiddo #1 put some actual time into prepping for the SAT (which he didn’t) his score might have been better. I don’t believe that the SATs have no value, nor am I against testing. If some schools think they are important in finding the students they want, they should use them. However, I don’t believe that the SAT offers the final word on student performance and potential.

1 Like

Clearly you took an extreme view here that is untenable and defending it is possible only by denigrating wide swathes of very intelligent, intellectual and quick professionals by contrasting them to your “decades” as a researcher. I also spent decades writing and researching, publishing book chapters, peer-reviewed articles, presenting at conferences and holding a patent. Still, I don’t wake up with a feeling of intellectual superiority or uber-intellectualism over any of the careers listed. And if you work in any kind of competitive field, the fear of being slow and being “scooped” by other researchers who are quicker than you is one of the biggest and most stressful aspects of any research career.

When you liken operating on a patient who is under general anesthesia, whose lungs and body suffer from every ticking hour, whose risk of infection increases directly with length of surgery with changing a timing belt on a '87 accord, I think you accomplish both a contortion of the absurd and unfortunately insult both the surgeon and the mechanic in your goal to defend your original statement.

6 Likes

Reminder that CC is not a debate society. Please move the back and forth to PM. Further posts on speed will be deleted without comment. Thank you.