@likestowrite, sorry to hear you want to leave us. Please reconsider. I don’t think @thshadow had any ill intentions.
Hey, I wish I could redo some of the messages I posted in discussions with @DoyleB and @suzyQ7.
Thinking about the footnote that was on the 2014 Guide and why it was removed from the 2015 Guide. Why remove it? One reason might be:
CB didn’t want people to doubt the SI% Table by calling attention to the fact it was from a research study.
In fact, if the footnote was left there, that would give them some leeway to explain away any discrepancies that would arise come September if the SF cutoffs didn’t match the SI% table.
They make it look like that document is final and not going to be revised.
@TallyMom2017, thanks for posting this SAT link. I do expect CB to release a report on PSAT at some point.
They are also going to produce a “synthetically derived” concordance table to link new SAT to ACT. Expecting report on that also.
After skimming over the SAT validity study, it is apparent that CB puts a lot of effort into their research products. I’m sure much planning went into the PSAT research study. They definitely have the ability to use the best statisticians around.
Wish I could explain why the psat concordance tables don’t seem to match the SI% table. Presumably they both came from the same research study.
Wouldn’t the concordance tables be concording ACTUAL current percentiles to historical? Perhaps that’s why they aren’t consistent with the page 11 SI percentiles. And we don’t know what the actual percentiles are . . .
EDIT/UPDATE: . . . or don’t we? If the condolence tables are actually concording real percentiles, then wouldn’t that give us a real percentile table for the SI’s? Just a thought. Perhaps it’s already been mentioned.
And btw regarding #2123 - that was supposed to be “concordance” not “condolence”. Although making a condolence table at this point doesn’t sound like a bad idea . . .
@Speedy2019 I had mentioned CB’s statistical resources earlier. It will be interesting to read through their PSAT validity. However, I thought the SAT validity link might give the statisticians among you a better idea about methodology, standard deviations and such.
That doesn’t help us with cutoffs but might give methods insight in the whole new PSAT 8/9, PSAT 10, PSAT/NMQT suite they are rolling out. I also saw that the PSAT 10 will be administered nationwide in February 2016, probably only in the states that have contracted with CB. This seems to be a shift in time, too …
@likestowrite – please don’t go - I really appreciate your perspective, many contrivutions and the efforts you put in to develop a chart of info to consider. Please stay with us. @thshadow I am SURE (right?) will do his best to be more sensitive to those of us who are not programmers, math whizzes or stats experts & who might not find it so easy to follow the approach he & others find logical to take.
Those of us who are stressed , everyone force a SMILE and take a slow deep breath , exhaling slowly. I do that, It helps me.
I’ve been skimming all the posts since yesterday, but I’m guessing my son’s high school doesn’t see a ton of NMSF students historically as it’s a smaller Iowa school (where the focus is on ACT), there was no special prep for PSAT and the announcement for the sign up was almost nonexistent. I can send the GC questions if someone can summarize what I should ask. It appears to be:
How many students with SI above 200 and how did those break down?
More or fewer 99% (user percentile) for SI and TS than you typically see?
Average total score for this year and last?
Is there anything else that would be helpful? No guarantees that I’ll get an answer, but I can try.
@IABooks – that’s great - thanks for joining in the effort to get more info. Not sure what state you are in - GC’s can get data for their students, the school & the state. If they have SIs & percentiles that is most informative. Ours only reported based on total score rather than SI but other CC’rs I believe got the SI info. How many took the test? How many are in the 99+%ile range if that is reported? Again, huge thanks!
I have a question. My understanding of statistics is minimal. But as I am driving around in my car picking and dropping off my kids, I keep wondering why the percentile table on p.11 is so far off from what I am hearing reported anecdotally. I decided to review some basics of a normal curve on my phone while I was at a stoplight. Isn’t is suspicious at the top end of the curve the percentiles so closely follow what would be expected from a normal distribution. For example, given that CB tells us at the bottom of page 11 that the mean of the curve is 148 with a standard deviation of 26 and the 99+% exactly starts at 214 which is so close to what a normal curve would predict. Maybe when preparing the percentiles, CB just assumed the upper end would follow a normal curve and didn’t have a enough data from research study. Although, that is unlikely given how large their studies are. Please forgive me if I am totally off. I just wanted to see what you guys thought who are so much better with statistics.
As I mentioned in another post, there’s an “obvious” way to reconcile the concordance tables and the percentile tables, without assuming any mistakes - simply that this year’s class has a lot fewer high achievers than last year’s class. Obviously, this is implausible on its face, but it’s mathematically logical. CB through its research study / crystal ball decides that someone who gets a 215 on this new test would also get a 215 on the old test. Furthermore, they decide (either through some prediction or maybe using some data from the actual test takers) that only 0.5% of test takers in this year’s class can get a 215 (vs. 1% that got a 215 in last year’s class).
I guess I’m just restating the same puzzle that we keep coming back to. Why would CB say that 215 this year is about equal to a 215 last year, except that only 0.5% of our kids will get (or did get?) a 215?
My concern is CB assumed that the top end of the bell curve (the top 1 to 2 percent) just simple followed a normal curve. They used the research data to roughly figure out mean and SD and applied this to the top end of the curve. As we all know, the top percentages on this type of test do not necessarily follow a normal distribution at all. CB made no attempt to reconcile the real data from what would be predicted at the top end and reported the 99+% at what would be predicted by the mean and SD. That possibly could be why the percentiles at the top end seems so off. CB was not really concerned about the top 1% when preparing page 11. Again, I don’t know much about statistics at all. I just think it is so suspicious that 214 as the 99+ percentile so closely fits what would be predicted from mean and SD.
@Speedy2019…That is great information. We are in Texas also. Keeping our fingers crossed until September.
@AJ2017 - that’s a really interesting idea. (I hadn’t read your first post when I posted above.) You’re right, 214 is 2.54 std devs above the mean, which would make it just about exactly 99.5%…
So the percentile table is strictly a bell curve, extrapolated from whatever data they had. And then the discrepancy (shown by the anecdotes) is that this group of students is less normally distributed than before. Maybe the kids that prepped did much better, and/or there were more kids that prepped than before…
And if the percentile table is just a bell curve (and maybe they didn’t really care about the upper end), possibly the concordance table was computed completely differently - possibly in a way that’s actually accurate at the top end.
That would agree with the anecdotes - that the percentiles at the top end are just wrong, and that the concordance numbers are correct.
What we really want to know from GCs: if the school usually has, say, 10 semifinalists, what is the 10th highest SI? Count tie scores among the 10.
I really feel that the percentiles at the top end on page 11 might be useless and very misleading, In this case, anecdotes might be more helpful in determining cut offs. Like other posters have recommended. Looking at large schools with consistently high number of NMSF. And specific data like the data provided from Illinois would be most helpful obviously. Just looking at the mean of a school may be misleading. Even a mean that is just 50 to 100 points higher than what would be expected could be hiding a number of high scores. My feeling is scores on this test at the high end will not follow a normal distribution at all.
And use the concordance, too. That seems to line up better with the anecdotal data, right?
@CA1543 and @MatzoBall Thanks. I think we’re going to find a lot fewer than 10 qualifiers on average. Lol. Maybe a couple a year. I’m plotting now how best to go about this to see what I can find out. A phone call from might work better than an email.
Separate question–I keep seeing references to 99+% at different SIs. If my son is 99+, is that indicated on his report? The info we got says 99 for all percentiles but he’s at 216 which is above some of the references to 99+ I’ve seen on here.
@thshadow: “As I mentioned in another post, there’s an “obvious” way to reconcile the concordance tables and the percentile tables, without assuming any mistakes - simply that this year’s class has a lot fewer high achievers than last year’s class.”
If I’m understanding what you’re saying … not just last year’s class. The concordance tables concord to PSAT scores from 2014 and prior to 2014.