USNews 2011 - New Methodology and Stanford's strange result

xiggi · 08-16-2010, 10:22 PM

This is not a thread to complain about the final rankings; there will be plenty of those.

This year, USNews introduced a few changes, including a change in the Peer Assessment. The resulting Academic Index seems to exhibit a rather strange behavior.

The definition of the new Undergraduate academic reputation index intimates that it is a weighted average of the Peer Assessment score and the High School Counselor score.

But is it?

MIT
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98

Yale
Peer assessment score (out of 5) 4.8
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 97

Stanford
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 93

Columbia
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.8
Undergraduate academic reputation index (100=highest) 93

Caltech
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 92

Penn
Peer assessment score (out of 5) 4.5
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 91

Inquiring minds would like to know what kind of weigh could be applied to the average to yield such results

Here's a simple table that shows the results could be as simple as adding the numbers:

School PA - GC - TOTAL
MIT 49 49 98
Yale 48 49 97
Stanford 49 49 93
Columbia 46 48 93
Caltech 46 46 92
Penn 45 46 91

However, we know that the PA at 15% is twice the weight of the GCs score that is 7.5%

So, let's present a very simple table that doubles the weight of the PA.

School PA - PA - GC - WEIGHTED TOTAL - ORIGINAL SCORE
MIT 49 - 49 - 49 - 98.00 - 98
Yale 48 - 48 - 49 - 96.67 - 97
Stanford 49 - 49 - 49 - 98.00 - 93
Columbia 46 - 46 - 48 - 93.33 - 93
Caltech 46 - 46 - 46 - 92.00 - 92
Penn 45 - 45 - 46 - 90.67 - 91

The numbers work pretty well, except for one school. One has to wonder what Stanford did to Morse?

PS
Harvard
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98

Princeton
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98

ewho · 08-16-2010, 10:34 PM

xiggi, if you do this:

(PA+GA)*100%

You will see all of them are correct, except Stanford's. Regardless the weights, Stanford's should be 98. Columbia's might be due to the rounding error.

Simply put, the twisted mind of Morse hates Stanford.

Professor101 · 08-16-2010, 10:59 PM

Xiggi, they increased the graduate rate weight from 5% to 7.5%. Stanford's four year graduation rate is unsually low (<80%) for a top tier school considering it has the second most inflated GPA (average 3.56) after Brown. I think that this drags it down quite a bit.

Ranking aside, Stanford lags behind in two important metrics for undegraduate quality: average incoming student SATs and graduation rate.

ewho · 08-16-2010, 11:03 PM

Quote:

I think that this drags it down big time.

Stanford is at its place, just Columbia is misplaced.

phuriku · 08-16-2010, 11:03 PM

Xiggi: If you could, please post the rest of the scores up to #12, Northwestern.

xiggi · 08-16-2010, 11:10 PM

Quote:

Xiggi, they increased the graduate rate weight from 5% to 7.5%. Stanford's four year graduation rate is unsually low (<80%) for a top tier school considering it has the second most inflated GPA (average 3.56) after Brown. I think that this drags it down quite a bit.

Ranking aside, Stanford lags behind in two important metrics for undegraduate quality: average incoming student SATs and graduation rate.

Sorry, but that is simply not the case. You are confusing two distinct metrics. The ARI is now 22.5% and the Grad Rate is 7.5%. However, they do not form one 30% metric.

The increase in the graduate rate weight from 5% to 7.5% is TOTALLY irrelevant to this metric. The Academic Rep Index is NOTHING else than the weighted average of the PA and the GC rankings.

FYI, I just computed the ARI for the top 11 LACs and I could reproduce the index by simply applying the same "formula"

Try it for yourself.

Quote:

Stanford is at its place, just Columbia is misplaced.

No, the error is in Stanford's number. It should be 98 and not 93.

ewho · 08-16-2010, 11:15 PM

Quote:

Stanford is at its place, just Columbia is misplaced.

No, the error is in Stanford's number. It should be 98 and not 93.

I was answering Professor101's question.

I agree with you: regardless the weights, Stanford's number should always be 98. Do they have yield numbers in their Premium Edition?

xiggi · 08-16-2010, 11:20 PM

The Mouth
Peer assessment score (out of 5) 4.3
High School Counselor score (out of 5): 4.7
Undergraduate academic reputation index (100=highest) 89

Coach K Landia
Peer assessment score (out of 5) 4.4
High School Counselor score (out of 5): 4.7
Undergraduate academic reputation index (100=highest) 90

ObamaU
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.5
Undergraduate academic reputation index (100=highest) 91

NU
Peer assessment score (out of 5) 4.4
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 89

In case there are still doubts about how ARI works.

School PA - PA - GC - WEIGHTED TOTAL - ORIGINAL SCORE
Dart 43 43 47 88.67 89
Duke 44 44 47 90.00 90
Chica 46 46 45 91.33 91
North 44 44 46 89.33 89

xiggi · 08-16-2010, 11:22 PM

"are the PA scores hidden or something?"

Fourth column in the Premium Edition.

PS Sorry, EWHO. I misread your post.

Professor101 · 08-16-2010, 11:30 PM

Sorry, Xiggi, I confused the issue.

Senior0991 · 08-16-2010, 11:42 PM

Though irrelevant, I wonder why Stanford's (four-year?) graduation rate is low. There's only like 25 or so people transferring out each year (because that's how many they let in). Does this factor in people who leave for a year or a quarter and then come back, because that seems somewhat common. Also maybe the rate is low because of athletes red-shirting, which is very common. I imagine a red-shirt is not common at Harvard or definitely not Caltech. Or, because Stanford has low grad rates because it is so great and students want to stay there an extra quarter or two?

Also, if this mistake is real and not xiggi misreading data (I'm sure he looked it over, though, beforehand), and one of the weights that Stanford lags it was increased this year, then maybe it is all an intentional effort to undermine Stanford's rank and clearly distinguish it from the traditional elite three. Also weighting guidance counselors undermines Caltech and Berkeley (probably also a low grad rate at these institutions). East coast bias haha?

iCalculus · 08-16-2010, 11:45 PM

Y'all are way too worked up over this. Fact is, Stanford is a great school (well-known by Californians), but not at the very top. There are other elite schools. I cannot believe you guys are breaking down US News' methodologies to find errors. There are better things to do with your time.

kwu · 08-16-2010, 11:52 PM

^ Well, considering the significant influence of these rankings on the general public, helicopter parents and prospective applicants, this is a big problem. These posters are simply trying to work toward a solution.

Professor101 · 08-16-2010, 11:59 PM

Quote:

Or, because Stanford has low grad rates because it is so great and students want to stay there an extra quarter or two?

LOL, that is the best argument that I have heard in a long time. Yea, Stanford has its own golf course and I assume that a lot of students don't want to graduate so that they can play golf or like Tiger Wood to flirt around.

kwu · 08-17-2010, 12:02 AM

^ You don't say--if my college were to continue providing my family need-based F.A., I would stay for another year or two.