@mathyone, was it 80K or 90K? Anyway, that number was 9-12 graders. As you can see, the way they did the research (assuming the field test is their “research”), the same students would have taken the old psat and new psat in the same year. And then the real PSAT in October. There is a linear sequence they can follow from old to new.
But have they accounted correctly for the natural increase that happens to scores from sophomore to junior year plus the study prep the students did for the real PSAT.
2- the SI% on the 2014 understanding your PSAT which uses the old definition A of percentile
3- the SI% on the 2015 understanding your PSAT on CB website which uses new def. B of percentile
4- My translation of Column 3 by changing the def. of percentile back to definition A
Look Guys. Suddenly the SI% charts from this year look much more like the ones from last year if you use the method of gathering all the scores that share the same percentile with the next lowest percentile. I know @thshadow and @DoyleB have disagreed with this method, but see my argument in #1973.
I am too busy right now to check Column #4 as a predictor of possible state cutoffs, or how close this is to testmaster and prep scholar, etc. I would love to check a few of the states by this and see though. I think the best way to find out where a state’s % usually sits within the percentile chart. For instance, it is usually mid 99th percentile or high 98th. Then see where it falls on column #4.
Any critique welcome…
PS I raised the 99+ a little higher this time around to reflect closer to the ratio of past divides. Perhaps the first 99+ should be 221? Not sure about this…
@Speedy2019 thanks for your post about the field test. The whole time I asked myself these questions: 1) who are these people participating in the research? 2) How & when was this research conducted?
Someone posted the below info on another thread for IL state (traditionally with a cutoff SI score of 2016), and it does not bode well. IMO this is the best indicator so far that the cutoff for mid-level states will be the same or higher. Maybe even for high scoring states.
For IL, I got the following from my daughters school (My daughter has 218 (1470) on 228 scale). Her school counselor gave me the averages, in her school they had 10 children scoring 218 exactly and 12 children scoring 219 to 228 and 33 children scoring 215 or higher. All these are Juniors only numbers from my daughter’s school.
Based on the above, I feel Illinois might go with 219 cut-off this year but there is still a hope as following is the actual data of my daughter’s school of past NMSF numbers from 2011 graduating class to 2016 graduating class:
So, if my daughter who has 218 has to make then her School will have 22 NMSF’s for 2017 graduating class. (Statistics on her side from the above Table) has her school in alternate years have crossed 20’s as they had 17 in 2016. Will 2017 have 22 from her School then she would be in based on the above actual data till 2016?
But what concerns me is 10 children from her School only scoring 218 exactly, so state wide how many children are in the same boat? This concern puts a negative opinion on my part that IL might be 219 cutoff this year on 228 scale! Keeping my fingers crossed!
For people asking about the next step, I believe that April-ish NMSC sends letters of inquiry to school principals to “identify any errors or changes in the reported eligibility of their high scorers”. I do not know whether a reply is required by the school at that point, or if the school only replies regarding students who have moved or have changes to their eligibility.
Several years back, letters were sent to students at this point, but they no longer do that.
On CC, we get info from homeschoolers and people who are allow to look at the list given to their principal to figure out the Commended cutoff. The NMSC does not “announce” the Commended cutoff.
(Also, the state reports will hopefully come out in February, which will give the number crunchers a bunch more fodder.)
I agree that’s the definition, though a bit hand-wavy. What percentiles are you trying to match up? User percentiles from the population that took this test vs user percentiles from the population that took the last test? National percentile today vs national percentile last year? Or are you trying to (theoretically) equate percentiles from the exact same population, i.e. get the same set of kids to take both tests, and line up their scores? Personally, I think the latter might be the actual definition - though they’re all approximately equivalent / estimates of each other.
Thinking more, I think some of what I’ve said makes more sense if you think that the percentile tables include some data from the actual kids that took the test - and the concordance tables don’t / were made in advance. I know they said that the percentiles are also / still based on the research study - but see below:
Here’s a story I could tell which isn’t definitely wrong… They do their research study in February, and create concordance and percentile tables. Then they give the actual test. After the test is given and graded, they clearly have the “answer key” for the percentile table they created. I could argue that they don’t yet have all the info they want for the concordance table - based on how they’re defining the percentiles they’re trying to equate. In particular, they want to correlate these scores with actual SAT’s (as well as ACTs). (Unlike other parts of the story, this is an actual fact - they have said they are waiting to release final concordance tables until they have some SAT results.) So, they decide to check / update (only) their percentile tables. Yes, I know they have stated that the percentile tables are still based on the research study. But maybe there were some factors in their model (like average gain on the PSAT from preparing for it) which they had to estimate. After the actual test, maybe they can make better estimates of those factors, and so the percentile tables get tweaked (even though technically they’re still based on the research study). They release the updated percentile tables, but they only release the preliminary / not-updated concordance tables. Hence they disagree.
Pros for the story:
The concordance tables are stamped preliminary, with an announced target final release date, while the percentile tables are not.
The percentile tables are actually a real easily-understandable thing. If 97% (or whatever) isn’t the commended cut-off, it will be clear that they are wrong. If they have released percentile tables which are nominally final, and in 6 months they are essentially proven to be wrong - that can’t be a good look for the CB. (For the concordance table, they have the CYA “preliminary” stamp.)
@websensation Thank you for sharing. Yikes, another anecdote of high scoring! It is making me want to switch my chart above. Right now I have the 99+s starting at 220. I would like to move that to 222. That would make IL, who has had a cutoff in the low to mid 99’s still be in the mid 99s if the cutoff is 218.
Has anyone reported that their child goes to a school that produces 10 to 15 NMSF a year and the GC shared with them that only 3 kids 211 or higher? A parent with a child who is in the top five of his class but the child scored 205 would probably follow this thread and post this info because they would be disappointed.
My experience at my child’s small school in FL is this. 3 to 6 NMSF/year historically. This year 12 students in 99% (user) so that would predict probably around 10 this year from this very small school. Just another anecdote from Fl.
How interesting that there is such a cluster at 218. You have 12 kids spread amongst scores 219-228, and 11 kids spread amongst 215, 216, and 217. But, ten spot on 218. Like a magical target number or something.
@websensation wrote:"For IL, I got the following from my daughters school (My daughter has 218 (1470) on 228 scale). Her school counselor gave me the averages, in her school they had 10 children scoring 218 exactly and 12 children scoring 219 to 228 and 33 children scoring 215 or higher. All these are Juniors only numbers from my daughter’s school.
Based on the above, I feel Illinois might go with 219 cut-off this year but there is still a hope as following is the actual data of my daughter’s school of past NMSF numbers from 2011 graduating class to 2016 graduating class:
IL cutoff from 1 years ago - 215. No place to look that up.
IL cutoff from 2 years ago - 215. That was the 3rd from bottom of the 99s.
IL cutoff from 3 years ago - 216. That was the 2nd from bottom of the 99s
IL cutoff from 4 years ago - 213. That was the 3rd from bottom of the 99s.
The 3rd from the bottom of the 99s in this year’s SI table is a 207.
Is there anyone here who believes the Illinois cutoff will be 207 this year? Anyone? Bueller? Bueller?
The SI table is massively wrong. Massively. I just can’t understand it at all.
Several discussions on the last few pages were referencing GA Walton data. Here is a very likely scenario. GA’s cutoff score can be at the same level as high cutoff states like CA and NJ this time. Why? Our student demographics have been steadily changing for many years now. Many skilled immigrants have settled down in metro ATL. GA ranked 10th in the nation for H-1B visa. I wouldn’t be surprised to see 220 SI cutoff. https://www.foreignlaborcert.doleta.gov/pdf/H-1B_Selected_Statistics_FY2014_Q1.pdf
@micgeaux They could make a lot of sense, but only if CB assumed the phantoms would perform poorly. However, in the total score tables, they assumed the phantoms performed pretty well. Unless CB decided to use different assumptions about the phantoms’ abilities in different tables, it makes no sense.
However, nothing I’ve seen in this year’s tables makes sense, so that’s as good a guess as any. For now, I suggest assuming the SI table consists of a population of 3.2 million kids, and all of them took the test (so to be in the top 16000, you need to be in the 99+% category). Then shift by 1 to compensate for the new percentile definition.
Essentially that makes 216 the equivalent of the bottom of the 99s in the old table. The 3rd of the 99s from the bottom would be 218, which would be the computed IL cutoff for this year. Then you can use the table and the results you get won’t be crazy.
They do their research study in February, and create concordance and percentile tables. Then they give the actual test. After the test is given and graded, they clearly have the “answer key” for the percentile table they created. I could argue that they don’t yet have all the info they want for the concordance table - based on how they’re defining the percentiles they’re trying to equate. In particular, they want to correlate these scores with actual SAT’s (as well as ACTs). (Unlike other parts of the story, this is an actual fact - they have said they are waiting to release final concordance tables until they have some SAT results.) So, they decide to check / update (only) their percentile tables. Yes, I know they have stated that the percentile tables are still based on the research study. But maybe there were some factors in their model (like average gain on the PSAT from preparing for it) which they had to estimate. After the actual test, maybe they can make better estimates of those factors, and so the percentile tables get tweaked (even though technically they’re still based on the research study). They release the updated percentile tables, but they only release the preliminary / not-updated concordance tables. Hence they disagree.