New SAT scores vs PSAT scores?

Cara_Brook · June 21, 2005, 9:21pm

2310 vs. 224

inseniarator · June 21, 2005, 9:30pm

PSAT: 201
SAT: 1930 (***?)

I was scoring 2110-2220 on all my practice SATs in the blue book, how did all you people with 190-210 PSAT scores prepare to raise your scores for the SAT?

Papa_Chicken · June 23, 2005, 8:45am

Update-- I have assembled the data and have begun to analyze. In addition to describing the data statistically like I have in the past (but I’ll have more like t-test/matched pair analyses), there are now enough data to breakdown further analyses to PSAT segments…I think this will help describe the potential for improvement based upon a certain starting PSAT range. Also, I’m not sure I have the will to construct another graph; its a hand crafted thing that took quite a bit of time before with less than half the data we have now…I’ll see.

So, give me a couple of more days to fit this work in.

PC

Papa_Chicken · June 24, 2005, 4:30pm

Dear posters: I will be posting my final report in the next 1/2 hour using multiple posts for various sections…please refrain from posting so I can get those sections in sequence.

Papa_Chicken · June 24, 2005, 4:31pm

June 2005
By Papa Chicken, a CC forum fan

CONTENTS (each section to be posted separately)

Introduction
Study Objectives
Compiled and Transformed Data
Analytical Methods
- Distribution, Matched Pair testing, & Correlation
Distribution of the Overall Dataset
Matched Pair Testing of the Overall Dataset
Correlation Analyses of the Overall Dataset
Analyses of PSAT Subsets
Conclusions & Advice

Papa_Chicken · June 24, 2005, 4:32pm

This thread started out of pure curiosity to test the familiar notion that an improvement could be expected in general between PSAT score and subsequent SAT score. March SAT results were becoming available in early April 2005 when I first posted the thread. Conveniently, the new SAT format using three reasoning tests in math, critical reading, and writing allowed for the first time a simple comparison to the PSAT of similar design. As the volume of data grew, it became evident that (1) an improvement from PSAT to SAT ON AVERAGE appeared significant, and (2) some simple statistical analyses could help in the assessment.

In addition, discussions on methods and data quality can be found in the latter portions of this thread, particularly page 7 and on.

As the thread matured and our questions became more focused, it was more evident to me that this analysis, albeit fun to do especially with such a growing dataset, does have its limitations. These include: (1) potential bias of self-reporting (e.g., perhaps more data couples with strong increases between PSAT & SAT were reported; I have no way of knowing.this is NOT a random sample); (2) limited data range (reporting CC scorers are at the very high range of those taking standardized tests); and (3) risk of an individual inferring their personal situation based upon grouped data, that is, SOME SCORES (PERHAPS YOURS) WILL NOT FOLLOW THE GROUP TRENDS. So, beware of these and other limitations. Nevertheless, I am encouraged that there are so many positive stories out there, and that significant improvement between PSAT and SAT does appear not only possible, but normal, for the group sampled.

Big caveat: I am a neophyte statistician. Although I certainly hope not, there may be errors in analyses or approaches; therefore please point them out to readers if you see problems.

I encourage any reader to post more discussions on limitations, analyses, or errors. As the raw data are summarized below, anyone can do their own analyses.

Papa_Chicken · June 24, 2005, 4:34pm

The study objectives are fairly simple:

<ol>
<li> Test whether PSAT scores and SAT scores from the same test-taker are significantly different,</li>
<li> Assess the pattern of those differences, using the starting PSAT score as a basis.</li>
</ol>

With the growth of data volume since June SAT scores were reported, a more detailed evaluation can now be performed on sub-ranges of starting PSAT scores.

Papa_Chicken · June 24, 2005, 4:35pm

A data couple is one test-takers PSAT score and their SAT score from testing since March 2005 (when the new SAT format went into effect). In selecting data couples from posters that reported more than one PSAT or SAT score, I used the junior year PSAT, followed by the first new SAT taken. [The purpose of this discrimination was to fine-tune the analysis to compare the official PSAT versus the first taken SAT of new format.] I do not know if the majority of data couples that did not make these test attributes known in their posts are truly of the desired character, but I suspect they are based upon the timing of this thread relative to the new SAT, that more posters have a junior year PSAT relative to earlier years, and the speculation that most people will report their official PSAT if they took the test multiple times. I did include at least one data couple from a poster who submitted a sophomore year PSAT and a sophomore year SAT.

Additionally, the listing below includes data couples reported on this thread, as well as several others I found on other threads. (I pulled in other data during the earlier stages of this thread to yield an even multiple of ten data couples, for what I thought at the time was the last analysis.) I discarded data couples that were not reported in the format I requested (unless I received clarifications from the author), and I limited the lower data range to a PSAT score of 160, the lowest PSAT decade (group of ten) with more than one submitted value.

Total number of data couples (N) is 151.

The DIFFERENCE between the PSAT score and SAT score from each poster required some simple transformation. [Note: per one posters comment, I have changed nomenclature for the statistic I now call DIFFERENCE; in prior analyses, I termed it as a generic VARIANCE, but because VARIANCE has specific meaning in the statistics world, I changed the name to hopefully avoid confusion.] Namely, the PSAT score was multiplied by ten to yield an adjusted PSAT score; this produced an adjusted PSAT score with the same range as the SAT. The adjusted PSAT scores are not included in the listing below, but the adjusted PSAT scores were necessary to calculate the DIFFERENCES, which are included below. The DIFFERENCE is simply = SAT adjusted PSAT.

The PSAT scores, SAT scores, and associated DIFFERENCES are sorted by PSAT score. Each line represents one data couple from one poster. Scanning this list is probably the best way to grasp the magnitude, variability, and patterns of DIFFERENCES.

PSAT / SAT / Difference
160 / 2100 / 500
166 / 1840 / 180
167 / 1870 / 200
169 / 1900 / 210
172 / 1960 / 240
176 / 1860 / 100
178 / 1860 / 80
179 / 1990 / 200
179 / 1910 / 120
180 / 1930 / 130
180 / 2180 / 380
182 / 2040 / 220
182 / 2070 / 250
183 / 2040 / 210
184 / 1950 / 110
187 / 2070 / 200
189 / 1990 / 100
189 / 2080 / 190
190 / 2260 / 360
190 / 2240 / 340
191 / 2060 / 150
191 / 2260 / 350
191 / 2330 / 420
191 / 2090 / 180
192 / 1930 / 10
192 / 1950 / 30
192 / 2110 / 190
193 / 2180 / 250
194 / 2110 / 170
194 / 2200 / 260
195 / 1920 / -30
195 / 2120 / 170
196 / 2090 / 130
197 / 1930 / -40
197 / 2130 / 160
197 / 2180 / 210
197 / 2190 / 220
198 / 2100 / 120
198 / 2300 / 320
198 / 2100 / 120
198 / 2200 / 220
199 / 2170 / 180
199 / 2260 / 270
200 / 2300 / 300
200 / 2140 / 140
201 / 2100 / 90
201 / 2140 / 130
201 / 2290 / 280
201 / 1930 / -80
202 / 2020 / 0
202 / 2190 / 170
202 / 2150 / 130
203 / 2140 / 110
203 / 2150 / 120
204 / 2270 / 230
204 / 2320 / 280
204 / 2320 / 280
205 / 1930 / -120
205 / 1940 / -110
205 / 2220 / 170
206 / 2130 / 70
206 / 2300 / 240
207 / 2110 / 40
208 / 2110 / 30
208 / 2260 / 180
209 / 2070 / -20
209 / 2120 / 30
209 / 2210 / 120
209 / 2120 / 30
209 / 2170 / 80
209 / 2230 / 140
209 / 2200 / 110
210 / 1960 / -140
211 / 2060 / -50
211 / 2240 / 130
211 / 2270 / 160
212 / 2180 / 60
212 / 2220 / 100
212 / 2280 / 160
213 / 2020 / -110
213 / 2200 / 70
214 / 2010 / -130
214 / 2100 / -40
214 / 2120 / -20
214 / 2330 / 190
215 / 1860 / -290
215 / 2120 / -30
215 / 2140 / -10
215 / 2150 / 0
215 / 2260 / 110
215 / 2390 / 240
216 / 2190 / 30
216 / 2230 / 70
217 / 2130 / -40
217 / 2230 / 60
217 / 2290 / 120
217 / 2310 / 140
217 / 2330 / 160
217 / 2360 / 190
217 / 2380 / 210
217 / 2280 / 110
217 / 2130 / -40
218 / 2340 / 160
218 / 2350 / 170
218 / 2380 / 200
219 / 2030 / -160
219 / 2150 / -40
219 / 2210 / 20
219 / 2290 / 100
219 / 2000 / -190
219 / 2360 / 170
220 / 2160 / -40
220 / 2190 / -10
220 / 2350 / 150
220 / 2290 / 90
221 / 2340 / 130
222 / 2250 / 30
222 / 2280 / 60
222 / 2340 / 120
222 / 2180 / -40
223 / 2220 / -10
224 / 2250 / 10
224 / 2300 / 60
224 / 2310 / 70
225 / 2130 / -120
225 / 2260 / 10
225 / 2300 / 50
225 / 2320 / 70
226 / 2310 / 50
227 / 2140 / -130
227 / 2140 / -130
227 / 2280 / 10
227 / 2310 / 40
227 / 2270 / 0
227 / 2330 / 60
229 / 2170 / -120
229 / 2360 / 70
229 / 2280 / -10
230 / 2280 / -20
231 / 2390 / 80
232 / 2250 / -70
232 / 2310 / -10
233 / 2320 / -10
233 / 2340 / 10
233 / 2390 / 60
233 / 2240 / -90
234 / 2200 / -140
235 / 2230 / -120
237 / 2400 / 30
240 / 2400 / 0
240 / 2260 / -140

SCATTER PLOT OF PSAT vs DIFFERENCE:
I did not re-plot the data as I have done once at N=47 (post 68), as this is an arduous task. The visualization depicted on post 68 still holds however, just with more dots in the same basic pattern: greater positive DIFFERENCE at the lower starting PSAT range, approaching zero DIFFERENCE around the 225-230 PSAT range.

Papa_Chicken · June 24, 2005, 4:36pm

To achieve the study objectives, the entire dataset, as well as PSAT subsets, were subjected to several standard types of statistical analysis. The complied data and calculated DIFFERENCE were first manipulated in Excel. Then the sorted dataset was imported to a statistics software package, JMP v 5.0.1. JMP is a windows version of the venerable SAS platform, well known in the statistics world.

DISTRIBUTION: The purpose of DISTRIBUTION analysis is to provide basic statistical descriptors to each variable within the dataset. This analysis is performed on each column of data, namely PSAT scores, SAT scores, and DIFFERENCE. Distribution analyses provide two basic sets of statistics for each variable of the dataset. QUANTILES describe the breakdown of the variable in percentages, such as median, 25% & 75% quartiles, and other commonly used deciles and percentiles. MOMENTS describe the mean (average), standard deviation, and upper & lower confidence levels of the variable. The import of some of these measures will be discussed in subsequent sections. Graphical depictions are a superior way to communicate the DISTRIBUTIONS; however, the limitations of posting on CC prevent their use herein, so only the numbers are presented. Also, several other simple totals are provided to describe the DIFFERENCE data, such as number of improvers & deprovers, i.e., data couples with positive and negative DIFFERENCES, respectfully.

MATCHED PAIR TESTING: This is essentially t-testing with two variables, PSAT score and SAT score. The purpose of this analysis is to test the hypothesis that the two variable sets are statistically different. The analysis tests whether, within 95% confidence limits (a commonly used range), the mean DIFFERENCE between the two variables is zero.

CORRELATION: The linear regression method is used to explore correlations between PSAT score and SAT score. This method yields a one-equation model and a correlation statistic. R2, the correlation statistic, basically describes the fit of the model to the data. [More discussion on this later.] R2 is a number ranging up to 1 (the best fit) that measures the degree to which two variables are related. The derived equation can be used to calculate the dependent variable, SAT score, with a given independent variable, starting PSAT score. [SAT score depends on PSAT score.]

Papa_Chicken · June 24, 2005, 4:37pm

DISTRIBUTION

PSAT
Quantiles

100.0% maximum 240.00
99.5% 240.00
97.5% 235.40
90.0% 229.00
75.0% quartile 220.00
50.0% median 211.00
25.0% quartile 198.00
10.0% 184.60
2.5% 168.60
0.5% 160.00
0.0% minimum 160.00

Moments

Mean 208.70199
Std Dev 16.64764
Std Err Mean 1.3547658
upper 95% Mean 211.37888
lower 95% Mean 206.0251
N 151

SAT
Quantiles

100.0% maximum 2400.0
99.5% 2400.0
97.5% 2390.0
90.0% 2340.0
75.0% quartile 2290.0
50.0% median 2190.0
25.0% quartile 2100.0
10.0% 1950.0
2.5% 1860.0
0.5% 1840.0
0.0% minimum 1840.0

Moments

Mean 2176.0265
Std Dev 137.96656
Std Err Mean 11.22756
upper 95% Mean 2198.2111
lower 95% Mean 2153.8419
N 151

Needless to say, both the PSAT and SAT data represent a very high scoring sample relative to the entire population of test takers.

DIFFERENCE

Quantiles

100.0% maximum 500.0
99.5% 500.0
97.5% 364.0
90.0% 248.0
75.0% quartile 180.0
50.0% median 100.0
25.0% quartile 0.0
10.0% -106.0
2.5% -144.0
0.5% -290.0
0.0% minimum -290.0

Moments

Mean 89.006623
Std Dev 131.65994
Std Err Mean 10.714334
upper 95% Mean 110.17713
lower 95% Mean 67.836113
N 151

Approximately three quarters of the 151 data pairs were improvers, with one quarter deprovers. Mean improvement was 148.5 SAT points (N=110), and mean deprovement was 78.4 SAT points (N=37). Four data pairs (N=4) showed zero DIFFERENCE, i.e., no change between adjusted PSAT score and SAT score.

Papa_Chicken · June 24, 2005, 4:38pm

As mentioned in Section 4, the purpose of this analysis is to test whether the PSAT scores and SAT scores are statistically distinct. For this analysis, adjusted PSAT was used instead of raw PSAT; I have not provided the adjusted PSAT values in the listing above; therefore, for those intrepid readers who would like to do their own analyses, just multiply the raw PSAT by ten.

Matched Pairs
Difference: SAT-adj PSAT

SAT 2176.03 
adj PSAT 2087.02 
Mean Difference 89.0066 
Std Error 10.7143 
Upper95% 110.177 
Lower95% 67.8361 
N 151 
Correlation 0.64034

The bottom line of this analysis is that neither the upper nor lower confidence level around the 89.0 mean DIFFERENCE spans zero, meaning that the two variable sets ARE statistically different. Hence, one can confidently say that there is a change between PSAT and SAT score. [Of course causality is another matter.]

Papa_Chicken · June 24, 2005, 4:39pm

NOTE: in the spirit of playing with the data, I am probably taking this analysis farther than I should. Please keep in mind YOUR scores will be contingent upon how YOU prepare and perform.it is not the destiny of these numbers that will control YOUR fate.

LINEAR REGRESSION
Simple least squares linear regression in essence produces a linear model which minimizes the errors of fitting all data to the model. That is, a straight line (represented by an equation with x and y variables) is fit on a PSAT:SAT graph which minimizes the distances between each discrete, plotted PSAT:SAT point and the line. As applied to this dataset, linear regression produced the following:

y = 5.3068x + 1068.5
R2 = 0.41

where y is the SAT score dependant variable, and x is the independent PSAT score variable.

Before I discuss the equation and model, the limited correlation first needs elaboration. The R2 (pronounced R-squared) value is a measure of correlation, and the 0.41 result is not great for modeling purposes. Meaning that only about 40% of the variability of SAT scores in relation to their PSAT pair can be explained by the equation. The value does show that there is a correlation, as the eye can discern also (if you had a graph).

Nevertheless, it may be helpful for estimating purposes to understand the equation. The equation can be used by plugging in a starting PSAT score and solving for SAT score. The following list illustrates the model input and results.

PSAT / SAT / Diff (SAT points)
160 / 1918 / 318
170 / 1971 / 271
180 / 2024 / 224
190 / 2077 / 177
200 / 2130 / 130
210 / 2183 / 83
220 / 2236 / 36
230 / 2289 / -11
240 / 2342 / -58

Note that the transition from gaining SAT points to losing them occurs between PSAT range 220 and 230 (2289 is less than 230*10), implying that there are more points lost starting from high range PSAT scores versus more points gained the lower the starting PSAT score.

Papa_Chicken · June 24, 2005, 4:40pm

To further explore SAT gains or losses from various PSAT subsets, the data were broken into PSAT decades (groups of 10 points) with sufficient data pairs to yield maximally robust statistics. The 210 PSAT decade, for instance, is the group of data couples with PSAT scores ranging from 210 to 219. The following PSAT decades were analyzed:

170 combined with 180
190
200
210
220
230 plus 240 (decade plus 1)

A way to interpret these statistics is to take your starting PSAT score, then figure your likely SAT range will be somewhere between the SAT upper & lower 95% confidence values, with associated change from PSAT in SAT points using the DIFFERENCE 95% confidence values.

Papa_Chicken · June 24, 2005, 4:41pm

Distributions

SAT

Mean 1995
Std Dev 91.125442
Std Err Mean 24.354299
upper 95% Mean 2047.6143
lower 95% Mean 1942.3857
N 14

DIFFERENCE

Mean 180.71429
Std Dev 81.189644
Std Err Mean 21.698845
upper 95% Mean 227.59179
lower 95% Mean 133.83678
N 14

Of the 14 data couples, all had positive DIFFERENCES, that is, there were NO deprovers in the group. The mean improvement was 181.

So, assuming this dataset is representative (a fairly big assumption), one could expect a PSAT starting score in the 190s to roughly yield an SAT score between 1942 & 2047, with an improvement of 134 to 228 SAT points.

Papa_Chicken · June 24, 2005, 4:42pm

Distributions

SAT

Mean 2131.25
Std Dev 114.59997
Std Err Mean 23.39262
upper 95% Mean 2179.6413
lower 95% Mean 2082.8587
N 24

DIFFERENCE

Mean 183.33333
Std Dev 116.12088
Std Err Mean 23.703075
upper 95% Mean 232.36688
lower 95% Mean 134.29979
N 24

Mean DIFFERENCE was 183 SAT points, similar to the 170+180 decades. Of the 24 data pairs, 22 were improvers and 2 were deprovers, averaging +203 and 35 change from adjusted PSAT to SAT, respectively.

Papa_Chicken · June 24, 2005, 4:43pm

Distributions

SAT

Mean 2157.931
Std Dev 110.01231
Std Err Mean 20.428774
upper 95% Mean 2199.7775
lower 95% Mean 2116.0846
N 29

DIFFERENCE

Mean 109.31034
Std Dev 113.76451
Std Err Mean 21.12554
upper 95% Mean 152.58405
lower 95% Mean 66.036638
N 29

Mean DIFFERENCE was 109, with 24 out of the 29 data couples showing improvement, one not changing, and 4 showing deprovement. For those improving, average improvement was 146 and for those deproving the average loss was 82.5 SAT points.

Papa_Chicken · June 24, 2005, 4:44pm

Distributions

SAT

Mean 2202.0513
Std Dev 131.11224
Std Err Mean 20.99476
upper 95% Mean 2244.553
lower 95% Mean 2159.5496
N 39

DIFFERENCE

Mean 47.179487
Std Dev 126.21929
Std Err Mean 20.211262
upper 95% Mean 88.095048
lower 95% Mean 6.2639267
N 39

Mean DIFFERENCE was 47 SAT points, with 24 out of 39 data couples showing improvement (mean = +130), one no change, and 14 data couples showing deprovement (mean = -92).

Papa_Chicken · June 24, 2005, 4:45pm

Distributions

SAT

Mean 2261.4815
Std Dev 70.476603
Std Err Mean 13.563229
upper 95% Mean 2289.3611
lower 95% Mean 2233.6019
N 27

DIFFERENCE

Mean 17.407407
Std Dev 76.740952
Std Err Mean 14.768803
upper 95% Mean 47.765117
lower 95% Mean -12.9503
N 27
Mean DIFFERENCE was 17 SAT points, with 17 out of 27 data couples showing improvement (mean = +63.5), one no change, and 9 data couples showing deprovement (mean = -67.8).

Papa_Chicken · June 24, 2005, 4:47pm

Distributions

SAT

Mean 2308.4615
Std Dev 70.927916
Std Err Mean 19.671864
upper 95% Mean 2351.3228
lower 95% Mean 2265.6002
N 13

DIFFERENCE

Mean -32.30769
Std Dev 73.388674
Std Err Mean 20.354356
upper 95% Mean 12.040639
lower 95% Mean -76.65602
N 13

Mean DIFFERENCE was 32 SAT points (on average a loss of points between adjusted PSAT and SAT), with 4 out of 13 data couples showing improvement (mean = +45), one no change, and 8 data couples showing deprovement (mean = -75).

Clearly it is, on average, not likely to follow stellarly high PSAT scores with equally high SAT scores.

Papa_Chicken · June 24, 2005, 4:48pm

DIFFERENCE means and improvements/deprovements by PSAT range

DECADE: / 170&180 / 190 / 200 / 210 / 220 / 230&240
DIFF MEAN / 181 / 183 / 109 / 47 / 17 / -32
Improv’s % / 100% / 92% / 83% / 62% / 63% / 31%
Improv’s avg / 181 / 203 / 146 / 130 / 63.5 / 45
Deprov’s % / ---- / 8% / 14% / 30% / 33% / 62%
Deprov’s avg / ---- / -35 / -82.5 / -92 / -68 / -75

SAT Distribution by starting PSAT range

DECADE: / 170&180 / 190 / 200 / 210 / 220 / 230&240
SAT MEAN: / 1995 / 2131 / 2158 / 2202 / 2261 / 2308
upper 95%: / 2048 / 2180 / 2200 / 2245 / 2289 / 2351
lower 95%: / 1942 / 2083 / 2116 / 2160 / 2233 / 2266

DIFF MEAN / 180.7 / 183.3 / 109.3 / 47.2 / 17.4 / -32.3
upper 95%: / 227.6 / 232.4 / 152.6 / 88.1 / 47.8 / 12.0
lower 95%: / 133.8 / 134.3 / 66.0 / 6.3 / -13.0 / -76.7

New SAT scores vs PSAT scores?

CONNECT WITH US