This thread stresses me out and confuses me more than any other one on this website
Personally, @Frankmeister , I suspect something is being lost in translation:
What definition of ânational percentileâ are you referring to? The PSAT score report had âNationalâ percentile and âNational Userâ percentile, which we on CC try to distinguish between. If you just asked her if itâs a national percentile, thatâs quite ambiguous. She probably just meant âyes, itâs the same percentile across the nation, itâs not state-specific.â
Thatâs different than her saying âYes, 99+ is where the 99.1 percentile startsâ. It sounds like you asked her if 99+ meant 99.5, and she wouldnât confirm that. It sounds like theyâre secretive about what 99+ means (who knows why). I think what she might have meant was - and forgive the subtle logic difference - was that everyone in 99+ has at least a 99.1%. But NOT everyone with a 99.1% is in 99+. IOW, âI wonât tell you what 99+ means, but itâs better than 99.â
I seriously doubt youâre going to get any ground breaking info from a call center and a âsupervisorâ. If you call back, youâll probably get different answers.
Maybe a GC can get more info from CB on the source of the SI percentilesâŠ
Ok - now more confused. So 99%+ may be 99.1% + and SI index is national % not user %. Iâm not sure about the SI national % for SI index, reason being that the SI index can be calculated by scores which have both a user and a national %. At the 99% range there is usually little difference between the two percentages, so if a testerâs SI is well into the 99% this would presumably apply to the user group (1.7 mill test population) as well as the national (3.5 mill). In other words, if CB is telling a tester your sub scores are 99% user, then it seems they are ârestrictedâ then by this definition to the 1.7 mill pool of testers. How can a tester be at the same time (letâs say 99.00% user per sub scores) in the 17,000th spot and the 34,000th spot? But maybe Iâm not seeing it rightâŠ
Iâll email the CB person who told me the state summaries would be delayed and ask if SI % table is National or User. It can be confusing because both are referred to as âNationalâ in CB docs.
I agree, ânationalâ seems confusing. As stated by others previously though, the SI table is not stamped âpreliminary.â Is it fact that this table is not actual user? Or presumption? Iâm confused. I would think the mean and standard deviation would be for actual test takers. As, many have said - They. Have. The. Numbers. I noticed weeks ago, a school in San Antonio - BASIS - released the numbers given to them by CB. It was for 8th, 9th and 10th grade. I donât know why they didnât release 11th grade but the tables for the other grades included school, state and national numbers. Including Mean and SD. So, I just donât get how page 11 wouldnât be actual numbers. Does anyone know if the table was published prior to October - when the actual test takers sat for the 2015 test?
I think the basic question we want answered is:
Is the SI % table the actual percentiles of the kids that took this test?
Weâre pretty sure the answer is ânoâ. If the answer is ânoâ, I guess you could you try to figure out if theyâre trying to estimate percentiles over all juniors or just over junior test takersâŠ
@thshadow CB confirmed on Twitter on January 19 that the SI percentiles are based on a study and not on the complete test-taking population.
@CollegeBoard Were real scores or research study scores used to generate SI percentiles in âPSAT: Understanding Scores 2015â?
The College Board
@CollegeBoard
19 gen
We conducted a research study of U.S. students in the studentâs grade (10th or 11th).
Saw this from todayâs e-mail at the Testmasters website:
Anthony asks (February 6, 2016, 7:30 a.m.):
âWhat do you think will be the cutoff for the national Hispanic scholarship in Arizona?â
Bill says (February 6, 2016, 11:06 a.m.):
âBecause the Hispanic cutoff score is usually much lower than the commended cutoff score, AND BECAUSE THE DATA FUNCTIONS SO WEIRDLY AT THE TOP AND LOW ENDS, we were not comfortable making a prediction to that specific cutoff. Sorry!â
Itâs possible that Testmasters will no longer hazard a guess for TX (or any state) if theyâre seeing data thatâs functioning âso weirdlyâ at the top end. Still would love to see what they have.
Speedy, Shadow, and others, thanks for your excellent dissection of this conundrum.
Suppose the SI Table is national, and the non-takers are low performers. This will double the distance to the top to change the table to be comparable to previous years. For example, 98.0% national is 96.0% users, 99.75% national is 99.50% users. In addition, there is one row adjustment for the new percentage definition (at and above vs. above). If we incorporate both changes, the table would look like this:
215 - 228 99% (No 99+ because there was no 99.75%)
206 - 214 98%
203 - 205
203 - 205 96% and 97%
201 - 202 94% and 95%
Is there a one week echo in these forums?
Hey guys, these are emails that I sent out to CBâŠ
On 1/19/16 I wrote: âIâm sure you have received many questions surrounding the PSAT scoring. Here is mine:
Are SI percentiles based on actual 11th grade test takers of the October 2015 tests or are they based on a research sample? The college board just states the percentages on the SI table without giving any explanation as to how these percentages were calculated.
Thank you for your response.â
On 1/22/16 this is what I received:
"Thank you for contacting the College Boardâs PSAT 8/9, PSAT 10, and PSAT/NMSQT Assessments.
We received your request in regard to PSAT/NMSQT Score Reports. We will be happy to assist you and we apologize for any inconvenience you may have experienced.
Please click on the link below for more information:
https://collegereadiness.collegeboard.org/pdf/2015-psat-nmsqt-understanding-scores.pdf
If you have questions or need technical assistance, please contact PSAT/NMSQT for Educators by phone at 1 (888) 477-7728 (toll free in the United States and Canada) or 001 (212) 237-1335, Monday through Friday, from 8:00 a.m. to 7:00 p.m. (Eastern Time), or by e-mail at psat@info.collegeboard.org.
Thank You,
Alma
Agent #1284314
The College Board Service Center"
My email on 1/22/16 to Alma:
"Alma,
I was aware of the understanding score pdf file that you attached below. However, that pdf doesnât answer my question. My question was what were the percentiles on the SI table based on? Actual test takers or just a research sample?
Please answer the question. If you donât have an answer, please escalate the question to someone who knows the answer.
There are a lot of people out there are wondering & upset at CB. Why did CB publish an SI table with percentiles and not say what the percentiles are based on. In the past, they didâŠ.so, why not this year? There are theories out there that all these percentiles are bogus! I think CB needs to address this soon. Many people are unhappy about how CB is handling this redesigned test and are talking about having their kids take the ACT instead of the new SAT."
This is what I got back on 1/22/16âŠthis last response is an automatic response that just floods peopleâs inbox.
"Thank you for contacting The College Boardâs program for the PSAT 8/9, PSAT 10 and PSAT/NMSQT assessments.
If you are contacting about K-12 Reporting, please visit our resources below for assistance with most frequently asked questions.
https://lp.collegeboard.org/help-resources-accessing-scores-psat-related-assessments
The College Board is a not-for-profit membership association whose mission is to connect students to college success and opportunity. Committed to providing the highest level of service to all of its educators, parents and students; a representative will respond to your inquiry.
Please do not send duplicate messages. Be sure to add @info.collegeboard.org to your address book or safe sender list to ensure that you receive e-mail messages from College Board."
I gave up on CB! And itâs sad that our kidsâ future are in these peopleâs hands!
Testmasters: âAND BECAUSE THE DATA FUNCTIONS SO WEIRDLY AT THE TOP AND LOW ENDSâ
Like I suggested, they are having difficulty understanding the data. Look at data one way, reach one conclusion, look at the data another way, second conclusion.
They said 205 was 99th % and 219 for Texas which is most likely contradictory. Changed stance from 210 commended to our models support 200-210.
@Plotinus - I agree that it was confirmed that the percentiles are based on a study (I had forgotten the source).
Which makes it clear that the rep that @Frankmeister was talking to didnât know what she was talking about (or something got lost in the translation).
Or I guess I just donât know what â1) the SI values were based off of this last PSAT and not previous psat resultsâ meansâŠ
@thshadow â you said: âWhich makes it clear that the rep that @Frankmeister was talking to didnât know what she was talking about (or something got lost in the translation).â
Well not really â The Understanding your score Report says this:
âNationally representative percentiles are derived via a research study sample of U.S. students in the studentâs grade (10th or 11th), weighted to represent all U.S. students in that grade, regardless of whether they typically take the PSAT/NMSQT. For example, a studentâs score in the 75th percentile means that 75 percent of the nationally representative group of U.S. students in the same grade would have had scores at or below that studentâs score. User group percentiles are derived via a research study sample of U.S. students in the studentâs grade, weighted to represent students in that grade (10th or 11th) who typically take the PSAT/NMSQT.â
@thshadow . Have you checked your messages?
âAND BECAUSE THE DATA FUNCTIONS SO WEIRDLY AT THE TOP AND LOW ENDSâ
Presumably they have at least 1 school with a giant glob of high-performing students. (For example IIRC in one of our anecdotes there was a school that had a ton of kids with exactly a 218 - more than with 217.)
It would be even weirder if some schools had this bimodal distribution, while others didnât. Assuming they have any statistical chops, simply being non-normal (certainly in a single school) is not that weird. The weirdest thing is when schools contradict each other.
Guys: In case you are still on the âNationalâ vs. âUserâ discussion:
The terms you are looking for are âNationally Representativeâ vs. âUser Groupâ. The first is supposed to include everyone in the grade, regardless of whether they would have taken the test. The second is supposed to represent just those who would typically have taken the test.
Please see page 6 of the Understanding Scores for more detail.
The correct data for the page 11 SI table would have been actual data from the 2015 PSAT. CBâs original tweet informed us that they based this table on the research study, not actual data. Therefore, the question is: did they compose the table from the âNationally Representativeâ group or the âUser Groupâ?
If they responded (to the poster who got confirmation from the CB manager) that they used the âNationalâ Group that could mean either one, as both are supposed to represent national populations of students.
I agree with your analysis @annMarie74. Do you think NMSC will make distinctions between varying 2014 SI concorded scores for students with the same 2015 SI? (e.g., as you mentioned, a 220 SI in 2015 concords anywhere from 220 to 225 SI in 2014. My daughter has a 221 SI in California with the three subsection components concording to a 225 in 2014 since she received scores of 38 (78 and 79 in 2014) in both reading and writing with a lower math score of 34.5 (68 in 2014)