There is another thread about this ACT writing issue – http://talk.collegeconfidential.com/discussion/comment/19281274#Comment_19281274 Several students here got higher marks on the rescored ones - was important bc of high composite but low essay score. Format of essay changed which likely is also contributing to many lower scores.
It is a real problem - hopefully ACT folks will focus on it. i hear some colleges may drop the ACT writing section as the SAT essay has become optional now.
I even heard that some colleges now are considering banning standardized testings such as SAT/ACT as a way to assess admittance. Rather, they may start focusing on high school transcripts, GPA and recommendations, college app, essay, etc…the holistic way.
This is a different topic, however I think this idea of not having a standardized testing come from the idea of escaping the trouble that students and parents go through. And some colleges trying to re-brand themselves. There need be some common academic yardstick that measure a students college readiness. There is so much variation between schools, school district, state and everyone tries to give their student the advantage to get college admission. Hope the idea of SAT/ACT like test going away occurs only with a better standardized test!
I wrote to Testmasters regarding the inconsistency between the SI chart and Testmasters’ analysis of the 10,000 TX scores. Here are the questions and replies:
Tim Keehan says:
February 10, 2016 at 4:18 pm
This is terrific analysis, thanks for your hard work. I do have a question. Your projection of 219 for TX seems difficult to square with the PSAT/NMSQT “Understanding Scores 2015” SI chart, which places 219 well into the “99+” percentile, which is based on a national research sample. The SI chart would seem to place TX at 215-217. Can you explain the inconsistency?
Michael says:
February 10, 2016 at 4:38 pm
Hi Tim! Thanks for your kind words! There’s two things to note about the Understanding Scores chart put out by the College Board: 1) as you said, these scores are from a national sample while National Merit is awarded on a state-by-state basis. 2) it’s unclear where the 99+ scores differ, meaning no one knows where each fraction of a percent lies. A 99.5% could be just at a 214 and a 99.6% at a 215, or it could be that 214-220 could all be 99.5%. Note that the broad range of 99+’s is actually a new addition by College Board because if you take a look at this archived copy of the 2014 Understanding Guide, on page 3, CB simply lists 224-240 as 99+ without detailing each individual score in that range. I hope these two points clear up the questions you had about our post and the Understanding guide!
Tim says:
February 10, 2016 at 4:54 pm
Michael, thanks for your prompt response. I believe last year’s NMSF cutoff for Texas was 220 (below the 99+ percentile of the 224-240 range you cited). The projected 219 for this year would put Texas a number of rungs into the 99+ percentile. I can’t figure out why last year’s scores (and previous years as well) has Texas in the 99 percentile, four rungs below the 99+ mark, and this year’s projection of 219 has Texas six rungs into the 99+ percentile. That seems to be the inconsistency. Could the SI chart be wrong or somehow skewed (which would seem odd since the College Board possesses all the scores)?
Michael says:
February 12, 2016 at 1:48 pm
Hi Tim, we can’t necessarily comment on how College Board’s SI table was created, since all we know is the data we have. But if we’re speculating here, it’s possible that the SI table functions different than previous years’ because some values, namely odd selection index scores, are much more difficult to attain due to the way the selection index is set up. Because you’re summing the Reading, Writing, and Math test scores and multiplying by two, the only way you’ll get an odd value is if you got a XX.5 on the math section, which occur slightly more frequently than XX.0’s, especially near the low end. This can be seen visually in our graphs which show a “sawtooth” pattern where there are large jumps in between discrete values. As a result, the large range of SI values might be a bit more tightly grouped than it initially appears.
Hope this helps!
Would be interested to hear thoughts from the board.
The thought behind going test-optional was to equal the playing field wasn’t it? That some with less access to private tutors etc. were at a disadvantage so could pursue an alternative path to a college admission. But isn’t the revised SAT supposed to address these issues? The test is more straightforward, assesses preparedness rather than “aptitude”, etc. Plus there’s a free tutorial (which from what I can see seems decent - at least my kid likes it). And I’m pretty sure ACT is looking into providing some sort of online learning thing as well because we were surveyed about that last summer.
@VABogart you had a great question for them but I don’t think they answered it directly. :-?
Math scores ending in .5 actually occur less frequently than those ending in .0, resulting in more even-numbered scores. There are many more even than odd permutations possible.
@MatzoBall yeah that was noted earlier on this thread - I’m wondering if there will be a lot of odd numbered cut-offs as a result.
@VABogart They didn’t answer your question directly. And, even if all the 214-220 were 99.5, Texas should still not be 219 (if it performs similar to past years). So, there would have to be another reason.
Have you guys finally figured out all the cutoffs? Ping me when you do…
@GMTplus7 Nope, but it looks like testmasters has the likeliest ones. Fantastic article from compassprep if your interested. Very thorough, very clear.
http://www.■■■■■■■■■■■■■■■/problems-with-new-psat-part-1-inflation/
@VABogart Your question: “I can’t figure out why last year’s scores (and previous years as well) has Texas in the 99 percentile, four rungs below the 99+ mark, and this year’s projection of 219 has Texas six rungs into the 99+ percentile.”
That is the issue! We can solve one rung of the problem by noting the new definition of percentile, but as for the rest? Test Master seems to suggest that the uncertainty about the rungs could account for it:
“A 99.5% could be just at a 214 and a 99.6% at a 215, or it could be that 214-220 could all be 99.5%”
So it’s just barely plausible that a 220 in TX last year could be equivalent to a 219 this year if the intervening rungs are very small percentages: 99.48, 99.49. 99.52 etc. I find it hard to believe, but it’s one idea.
@suzyQ7
thanks for the link
This is why, long ago, I gave up trying to divine the new cutoffs: all the new PSAT scores are indexed vs a fantasy focus group.
@LadyMeowMeow others may disagree, but the fact that testmasters has 10,000 actual scores AND they know what they are doing AND when you have actual SI numbers the math for cutoffs is really simple, I think their evidence is the most powerful in this thread. “Past performance is not indicative of future results”. Testtmasters, for Texas, has the actual results.
@suzyQ7 I agree. I don’t think testmasters would have published a 3rd bite at the apple unless they were confident. I’m just trying to square their results with the CB SI % table. It’s not easy.
What score on the SAT is required to ‘confirm’ National Merit Semifinalist? Is there a specific number?
@suzyQ7 I agree that Testmasters has provided the best info so far. There is one item that might cause me to question their results. If the 10000 data points they provided are all from HISD, and HISD has a specific program (with Testmasters) in place to boost the number of NMSF… perhaps they were very successful this year? If that’s the case, it might be difficult to extrapolate the results in HISD to the rest of Texas…or nationally. I am grasping at straws.
@bucketDad I don’t believe they have a meaningful number of their own students in the mix. They are trying to cover themselves so that no one figures out what high school (s) gave them the data. They know VERY well that their students would skew results (since they are just as smart, if not smarter, than the rest of us) so I believe that if they used their own students, they are not enough of them to move the needle. I bet they only used their own students to confirm the results they already saw from the real data they got from the school(s).
“I don’t believe they have a meaningful number of their own students in the mix.” - @suzyQ7 - I doubt they had a choice. They got a dump of a ton of students from a school district, and they analyzed that data. If they had prepped a high (or low) fraction of those students, there’s not much they could do.
@bucketDad - yes, the only plausible explanation for this prediction to be wrong is if this school district ends up with way more NMSF than last year. Looking at their chart, there are about 10 kids at each score in the 210’s. So moving it down 3 points to 216 would increase NMSF from 60 to about 90 for that school district. A 50% increase (for a big set of students) is certainly possible - but probably not likely.
@suzyQ7 I don’t think their results are biased with their own results. I believe they are using the raw results they received from their source. Let’s assume the source was HISD. Somebody in this forum posted a link to an article that described a recent program in Houston where the expressed purpose was to increase NMSF in the district. The program was targeted towards the best students. I could imagine that if that program was particularly successful (compared to prior Houston performance) it could result in the upper end bump we see in the Testmasters data…and it might be difficult to extrapolate those results across Texas if that were the case.
Oh, and in case anyone comes to this thread late, here is the testmasters link we’re all talking about:
http://collegeadmissions.testmasters.com/update-2-2016-national-merit-semifinalist-cutoff-score-texas-estimate/