<p>The NRC ratings don’t work well enough to support the kind of analysis PG2000 tried to do. I wish they did, but they don’t. The “S” rankings were determined by giving people a list of twenty or so objectively measurable features of graduate programs, and asking them what relative weight they would assign to each in evaluating the program overall, and then applying those weightings in a sophisticated way to the data from each program. The problem is that the most important things to know: who gets the best students, what faculty are the most intellectually exciting and have the most leadership? – aren’t on the list of measurable data. So the “S” rankings, while interesting, get kind of random because they are missing the content.</p>
<p>The “R” rankings attempt to correct for that by asking people to rank which schools they admire most from a limited set of choices. Then, there was an attempt to take all of those rankings of schools by people with different sets to choose from and correlate them with specific categories of data. Well, guess what? R rankings seem to be based on student GREs and institutional wealth. Whoop-de-doo.</p>
<p>When you put 'em all together, they don’t mean very much, except that the top universities in the country are still the top universities in the country. Not terribly surprising. But trying to use those metrics to figure out who is better than who else, and by how much, is like trying to rate athletes based on the length and resilience of their toenails.</p>
<p>PG’s method also assumes that all intervals are created equal, that #4 compared to #1 is three times worse than #2 compared to #1, and that #10 is 10 times worse than #1. Just not true.</p>
<p>Also, the numerical reductions for these universities mean different things. It’s amazing that MIT does so well, given that it doesn’t offer half of the subject matters at all. Chicago and Yale are hobbled by their lack of engineering, total or relative, depending on which you are talking about. Out of the 32 areas, maybe a fifth just aren’t relevant for Chicago. At the same time, by not acknowledging the contributions of law, medicine, education, and business school to the intellectual life at a university, you lose some of what makes many of them special.</p>
<p>This isn’t a whine about Yale vs. Chicago, or Harvard vs. Yale. They are all among the 20-30 truly great universities in the world. Based on breadth and depth, some of those are doubtless a little greater than others. I, for one, would have no trouble saying Harvard is greater than Chicago, as if that mattered. That doesn’t mean that Harvard’s X Department is greater than Chicago’s X Department, or that a single X student at either could perceive the difference, or that any difference was actually significant. It would mean that Harvard does a few more things fabulously well than Chicago does, is right at the top in more fields, and where it isn’t the absolute best it tends to come a little closer. But why that should matter to anyone other than the Presidents, Provosts, and Trustees of the two institutions is beyond me. It doesn’t speak to the experience any individual student or faculty member would have with either.</p>