You’ve linked this twice as suggesting the original study was flawed, so I wanted to respond to it.
As a former quant, I used regression all the time. Linear regression is a very useful tool, as useful as a drill for someone doing woodworking. One of its advantages is that once a person understands the limits of the method, it is very easy to use, computationally fast, and quite often provides useful information.
But just as you need more than a drill to do woodworking, you need more than linear regression to perform analysis. In woodworking sometimes you need a drill, but other times you need a hammer, saw, chisel, screwdriver, clamp, etc.
Saul Geiser’s response talks about omitted variable bias. He runs a regression that suggests that adding student demographics makes HSGPA a more powerful predictor than SAT/ACT scores. Note that he doesn’t say that the effects of the SAT go to zero, just that the HSGPA becomes a stronger predictor than SAT/ACT scores. In fact, after adding high school demographics, both have a strong predictive effect.
One of the key limitations of linear regression is that it assumes that it assumes linearity, and very often the world doesn’t work that way. In the real world, practitioners will look at what linear regression suggests and use other tools to verify what it is suggesting. But quite often academics don’t do that, and that’s apparent for those that have read a lot of academic papers.
What I liked about the STTF report is that they didn’t stop at just linear regression. They also looked at the interaction of SAT scores and GPA visually, using a number of controls that are related to student demographics, including race and income before coming to their conclusions. Here is one such example from the report.
