It was more a complete overhaul than a tweak. The first few rankings were entirely based on a survey USNWR sent to college presidents, asking them to name the top undergraduate colleges. The top 5 USNWR ranked colleges based on this survey were something like below.
- Stanford
- Harvard
- Yale
- Princeton
- Berkeley
After the college rankings articles started to gain popularity, USNWR hired statisticians to come up a more scientific looking formula for ranking colleges. The first guy they hired created a formula, which ranked Hellenic College of Theology #1. . Hellenic College of Theology had a high expenditure per student, driving up its ranking.
USNWR didn’t think this ranking would go over well, so they hired another statistician to create a different best college formula. The 2nd formula used 5 metrics to rank colleges and added up the scores of those metrics – SAT scores, acceptance rate, yield rate, expenditure per student, and % of faculty with a PhD. I believe SAT scores had the highest weighting. The new rankings had HYPSMC as #1 to #6, which USNWR apparently approved of, although some might consider the order unexpected with Yale #1 and Harvard below Caltech. The new ranking formula hurt publics like Berkeley, which dropped from #5 to #24. And helped selective privates with large endowment per student, particularly smaller ones like Caltech that didn’t do as well on the college presidents survey, which increased from #21 to #3.
In the following year USNWR had another major change and switched to the following weightings, which return an academic survey and add in a small weighting for5-year graduation rate. The new weightings yielded more changes in the rankings. With the peer survey returned, Berkeley had another large change – jumping from #24 to #13, and Caltech dropped below Harvard. The top 3 were Yale (#1) , Princeton (#2), and Harvard (#3).
25% – Academics Rank Colleges on Marginal=1, Distinguished=5 Survey
25% – SAT Scores, Acceptance Rate, Yield (“Selectivity”)
25% – Faculty Ratio, % of Faculty with PhD, Faculty Salary, … (“Faculty”)
20% – Expenditure per Student
5% – Graduation Rate
Throughout the 90s, USNWR made smaller changes to the rankings, which might be considered tweaks – things like adding more weight to graduation rate and adding in an alumni giving weighting. By 1996, the “best college” weightings had become more stabilized, with very few changes in the next few years. These more stable weightings produced the following “best colleges.” At this point, the college rankings edition of USNWR were huge sellers, by some estimates earning millions each year.
- Harvard
- Yale/Princeton
- Stanford
- MIT
- Duke
- Caltech/Dartmouth
- Columbia
- Chicago ...
- Berkeley
It should be obvious that the weightings are completely arbitrary and are not a meaningful measure of the “best colleges.” The best colleges for any particular student is likely to be completely different from the ranking on such lists.