@OHToCollege I don’t understand. IF the majority of the 10,000 student paid for test prep with Testmasters, how is the sample unbiased? I’m guessing the overwhelming majority of students taking the PSAT in Texas didn’t pay for test prep…perhaps that’s wrong.
@CA1543 Quite a few of those schools are private - St. Johns; Kinkaid; Strake Jesuit;
Bill says:
February 9, 2016 at 12:19 pm
Ethan,
We are confident that this projections is accurate. However, we do want to stress that this is still just an estimate. The actual cutoff score could be a point or two higher or lower – we just won’t know until the fall. At the very least, I can assure you that this data set did not include any type of bias toward high scoring students.
If Testmasters surveyed some of their own students, there is bound to be some bias in SI scores because a strong argument could be made that their students, who paid for their services, probably obtained higher scores than other “normal” TX students. Or maybe they also obtained scores from other students and somehow concorded their students’ scores. Anyway, I do tend to believe what Testmasters says.
Good confirmation @DoyleB! If their data is also construed to be nationally representative, the simplest thing we can say is that the expected SI cutoff for a state tends to stay close to the previous year’s cutoff, give or take a point or two. TX goes from 220 last year to 219 this year, not adjusting for scale differences. Thus, OH 214 and commended 199-200. Take this with a large pinch of salt.
But if we build a percentile table based on their data, the TX cut off score comes out at the 99.41 percentile. Isn’t it higher than the historical results for TX?
@AnnMarie74 You are right, but this could mean for the new PSAT, better test takers did BETTER, so there was a greater gap. Basically, the test did not do a good job of differentiating among top test takers, so they were bunched together.
How do you get 99.41 percentile?
I know this sounds silly, but they would have to be careful not to double count students. For example, if you get the computer download of all the students in a district, you would have to take out any of the Testmasters students that are already included in the download. But, I’m guessing they accounted for that.
@bucketDad, if they had an inherent bias in their sample, they would have indicated higher number of SF in their sample in relation to appx. 227K TX juniors who take the test. Instead, they chose to leave it at 60, perhaps to be conservative in their estimate. If I had to choose, I would choose an SI on the lower side of 219 rather than higher. Unless, of course, you all believe there are better testing organizations out there who do a fabulous job compared to Testmasters!
@websensation Sorry, friend, but I am not seeing it. If last year, top 0.7% of TX test takers became NMSS, why only 0.6% this year?
@AnnMarie74 Where are you getting that 0.7% of Texas test takers became NMSF last year? 1353/227000 = 0.596%
I got the 1353 number from @Speedy2019. Is that number incorrect?
Testmasters just posted the following question and answer concerning the anticipated NM cutoff for California:
Question:
“Thanks so much for doing such great work! Was wondering whether you still believe California’s cutoff will remain at 219 since you have now increased Texas’ SI from 217 to 219? Do you think a California SI of 221 still stands a good chance of making the cutoff in light of the data that you have analyzed?”
Reply
Bill says:
“A 221 should still be sufficient for nmsf status in California.”
@OHToCollege Thanks for the response. I was thinking that their choice of 60 was based on “historic data that of these 10,000 students, approximately 60 will become National Merit Semifinalists”. If their sample is biased with test preppers, the cutoff of 219 is higher than it should be. If they brought in enough other data to keep their sample unbiased, then their prediction is good. Based on the information they’ve included in the blog post and the response to the questions they’ve been asked, I can’t conclude whether their sample is good.
@DoyleB, the 1353 number is from 2014 annual report here http://www.nationalmerit.org/annual_report.pdf
According to this, there were 199,383 juniors who took the test, thus 0.67%.
@DoyleB Trying to estimate based on last year’s SI percentile table and the 220 cutoff. Not very precisely, perhaps.
Testmaster must have struggled with this as well. Too many students at 218 SI (14 out of 10K) to be included in the SF list, so as not to exceed the 0.67% ratio of SF to TX junior test takers. So they must have erred on the side of having fewer SF as opposed to more than historical. Ah! What to do with those 280 or so students expected at 218 SI’s!
@PicoLA I have a feeling that CA cutoff will be right at 221 but not very sure.
@OHToCollege That’s interesting. The state reports show Texas had 227,000 last year, and 217,000 the year before. Weird.
There’s probably a few that don’t quite meet other requirements of CB/NMSC, although a big drop between 217K and 199K. I would take the official count listed in their annual report. The 1353 number is an approximation based on number of graduating seniors in TX in relation to the 16K national SF count. That’s likely not going to change this year just because more or less than 199K students took the test.