U.S. News & World Report Announces the 2021 Best Colleges Rankings

[TennisParent]

@Data10

@bluebayou

But, that begs the question: is/was the methodology tweaked to ensure HYP rules the roost?

When the rankings first came out in 1983, Stanford was #1 for three straight years, but quickly dropped to #6. It did get back to #2 in '91, but has been struggling to get back there ever since.

Also back in '83, Michigan and Illinois were top 10 and ranked higher than MIT.

Cal Berkeley was #5, and then dropped into the 20’s three years later.

There was some interesting methodology tweaking going on in Year 4 of the report:

It was more a complete overhaul than a tweak. The first few rankings were entirely based on a survey USNWR sent to college presidents, asking them to name the top undergraduate colleges. The top 5 USNWR ranked colleges based on this survey were something like below.

1. Stanford
2. Harvard
3. Yale
4. Princeton
5. Berkeley
   

After the college rankings articles started to gain popularity, USNWR hired statisticians to come up a more scientific looking formula for ranking colleges. The first guy they hired created a formula, which ranked Hellenic College of Theology #1. . Hellenic College of Theology had a high expenditure per student, driving up its ranking.

USNWR didn’t think this ranking would go over well, so they hired another statistician to create a different best college formula. The 2nd formula used 5 metrics to rank colleges and added up the scores of those metrics – SAT scores, acceptance rate, yield rate, expenditure per student, and % of faculty with a PhD. I believe SAT scores had the highest weighting. The new rankings had HYPSMC as #1 to #6, which USNWR apparently approved of, although some might consider the order unexpected with Yale #1 and Harvard below Caltech. The new ranking formula hurt publics like Berkeley, which dropped from #5 to #24. And helped selective privates with large endowment per student, particularly smaller ones like Caltech that didn’t do as well on the college presidents survey, which increased from #21 to #3.

In the following year USNWR had another major change and switched to the following weightings, which return an academic survey and add in a small weighting for5-year graduation rate. The new weightings yielded more changes in the rankings. With the peer survey returned, Berkeley had another large change – jumping from #24 to #13, and Caltech dropped below Harvard. The top 3 were Yale (#1) , Princeton (#2), and Harvard (#3).

25% – Academics Rank Colleges on Marginal=1, Distinguished=5 Survey
25% – SAT Scores, Acceptance Rate, Yield (“Selectivity”)
25% – Faculty Ratio, % of Faculty with PhD, Faculty Salary, … (“Faculty”)
20% – Expenditure per Student
5% – Graduation Rate

Throughout the 90s, USNWR made smaller changes to the rankings, which might be considered tweaks – things like adding more weight to graduation rate and adding in an alumni giving weighting. By 1996, the “best college” weightings had become more stabilized, with very few changes in the next few years. These more stable weightings produced the following “best colleges.” At this point, the college rankings edition of USNWR were huge sellers, by some estimates earning millions each year.

1. Harvard
2. Yale/Princeton
3. Stanford
4. MIT
5. Duke
6. Caltech/Dartmouth
7. Columbia
8. Chicago ...
9. Berkeley

It should be obvious that the weightings are completely arbitrary and are not a meaningful measure of the “best colleges.” The best colleges for any particular student is likely to be completely different from the ranking on such lists.

@data10, just curious do you know how many years Princeton as been ranked the #1 university and who was ranked #1 before Princeton became #1? Was wondering if Harvard or Yale have ever been #1? I recall Stanford was #1 in during the 1980s. Would be interesting to see the history and how schools have shifted over time.

[Data10]

@TennisParent

@data10, just curious do you know how many years Princeton as been ranked the #1 university and who was ranked #1 before Princeton became #1? Was wondering if Harvard or Yale have ever been #1? I recall Stanford was #1 in during the 1980s. Would be interesting to see the history and how schools have shifted over time.

My post above mentioned that Stanford was #1 in the initial rankings based on college president survey, then Yale was #1 in the first ranking with weightings, then Harvard was #1 for several years after the rankings were tweaked. Princeton was first ranked #1 later… in 1998 and has been regularly been ranked #1 since then.

I believe one of HYP have always been ranked #1 in every year since they started using weightings, except for 2000. In 2000, USNWR standardized their variables, which led to Caltech jumping to #1, ahead of HYP in #2 to #4. After much criticism from confused readers, USNWR modified their expenditure per student formula in a way that disadvantaged Caltech and applied an adjustor on statistical outliers, like Caltech .This returned HYP to #1 to #3 in the following year, above Caltech at #4.

[ucbalumnus]

quote=“bluebayou;c-22939070”

[/quote]

@Data10

It should be obvious that the weightings are completely arbitrary and are not a meaningful measure of the “best colleges.”

Or is it that they are trying to reverse engineer a ranking that approximates popular perception?

[SJ2727]

I’m reminded of that old saying about torturing the statistics until they confess.

[Data10]

@ucbalumnus

quote=“bluebayou;c-22939070”

@Data10

It should be obvious that the weightings are completely arbitrary and are not a meaningful measure of the “best colleges.”

Or is it that they are trying to reverse engineer a ranking that approximates popular perception?

[/quote]

That’s part of it. I also think USNWR rankings have become well known enough that they can also have a strong influence on what popular perception is, particularly among those choosing to participate in the USNWR survey, creating a self-perpetuating system. This is more significant for rankings of smaller schools or specific departments within schools.

For example, I previously mentioned that Williams has been the number 1 ranked LAC for each of the past 18 years. The top 3 are almost always Williams, Amherst, and Swarthmore… in that order. Some of the “academics” who choose to fill out the USNWR 1 = marginal to 5 = distinguished peer survey are influenced by seeing them top the rankings each year, increasingly the likelyhood that they’ll mark Williams, Amherst, and Swarthmore as “distinguished.” This peer survey is the highest weighted component of the rankings.

The peer survey results are not free. When a CC poster listed them in 2013, the survey results were as follows. Bowdoin, Middlebury, Pomona, and others are unlikely to break in to the top 3 until they can overcome this large gap in their ranking on the “marginal” / “distinguished” peer survey. So I expect the top 3 to remain the same in future years, unless USNWR makes substantial methodology changes.

Williams – 4.7 out of 5
Amherst/Swarthmore – 4.6 out of 5
– Large Gap –
Bowdoin / Middlebury / Pomona / Carleton / Weselley – 4.3 out of 5
Various others – 4.2 out of 5

[merc81]

@Data10

So I expect the top 3 [LACs] to remain the same in future years, unless USNWR makes substantial methodology changes.

For mathematical reasons, Swarthmore, with an overall score of 95, could, even in the near term, tie or be overtaken by one or more of the five colleges with current overall scores in the 93–94 range.

[Data10]

@merc81

@Data10

So I expect the top 3 [LACs] to remain the same in future years, unless USNWR makes substantial methodology changes.

For mathematical reasons, Swarthmore, with an overall score of 95, could, even in the near term, tie or be overtaken by one or more of the five colleges with current overall scores in the 93–94 range.

Fair enough. Williams and Amherst have consistently taken the #1 and #2 spots since 2004 and still appear to have a good sized lead over others, based on overall score. However, the gap is smaller for Swarthmore, and in multiple recent years Swartmore has tied for 3rd with another college.

Williams – Overall Score = 100
Amherst – Overall Score = 98
Swarthmore – Overall Score = 95
Several others – Overall Score = 93-94

The point still remains that the rankings are perpetuated by influencing which schools participating “academics” choose to mark as “distinguished” in the USNWR sponsored survey. Had there not been such a gap difference between the portion of participating survey respondents marking Swarthmore as “distinguished” and Pomona/Wellesley/Bowdoin/CMC/Carleton/… , as “distinguished,” the latter would have overtaken Swarthmore in the current rankings.

[merc81]

@Data10: If you haven’t done so already, you may want to read “The Impact of U.S. News & World Report College Rankings on Admissions,” an analysis that found evidence for the recursive aspects of USN rankings along lines related to the ideas you have discussed in recent posts.