Northwestern President fired the first shot with his Op-ed
http://www.chicagotribune.com/news/opinion/commentary/ct-perspec-college-rankings-0828-story.html
Now USNews has shot back with this
âGraduation rates, retention rates and graduation rate performance are the most heavily weighted factors in the methodologyâ
They need to rethink this. If you have a very high gradation/retention rate, north of 95% your college is probably not challenging.
They should give 5% of their rankings to a âdifficulty/challengingâ metric.
If you have colleges A and B that have identical student profiles, but college B graduates 90% and A 97%⊠college B is probably graduating a higher quality student.
USNWRâs response is not very convincing. How do they âknowâ majority of schoolsâ data are accurate?
There are flaws in graduation/retention, but it is by no means about a lack of challenge. Those rates are often highly affected by people who drop out for financial reasons, disproportionately affecting schools with more low income students.
Look at top challenging schools too, and dropouts are not about the challenge but how well the school supports its students when the times get tough. A good school should not have a high drop out rate, but perhaps it has a lower average GPA.
The list above is full of challenging schools that no one would argue (in addition to grade inflated ones too of course, but stull). What you will find is that if you compare this with the NYT stats on college economic diversity, I suspect thereâs a strong correlation.
@Greymeer Or college B provides better financial aid or college B provides a better student life experience or college B provides etc.etc.
The Op-ed (which is excerpted from a book by Northwesternâs president and a professor there) comes off not so much as an attack on US News but more like Northwestern whining about all the other colleges being dishonest and gaming the system. They go through a number of dubious practices that a school could do or âallegedlyâ has done without naming any names, and without stating how widespread these things are, or how much effect they really have.
Sure there is some gaming of numbers. And sure someone could question US Newsâ priorities or methodology. But this is a pretty weak attack on the ranking system. Of course, it could be that anyone who frequents this website will read it and say, âYeah, somebody somewhere is probably doing that. So what?â
âSure there is some gaming of numbers.â
How do you know that there isnât a lot of gaming of numbers? The incentives are there for everyone to game their numbers.
âAnd sure someone could question US Newsâ priorities or methodology.â
Certainly. Many people on CC and elsewhere have.
âBut this is a pretty weak attack on the ranking system.â
Weak in what way?
If you want to see more effective criticism, look to Reed College. Reed is currently the only top school that refuses to cooperate with USNWR at all, and doesnât provide USNWR with the data that they need for their ranking system. https://www.reed.edu/apply/college-rankings.html
Of course, USNWR ranks Reed anyway, they just substitute very conservative numbers for the missing values. This ensures that Reedâs ranking stays low.
At one time, Reed was in the Top Ten of the USNWR National LAC ranking. Then Reed criticized the system and stopped supplying data. This is what happened next:
So the President of Northwestern has to be careful here. He may criticize USNWR, but he will probably continue to cooperate with them.
US News doesnât have the resource to know the âmajority of data are accurateâ. So what even if, say, five of the seven variables are accurate? Manipulating two of them is enough to change the ranking. A while back, someone on CC caught the misreporting of NAE number by USC. Their engineeringâs ranking dropped few spots after correction. Recently, I caught what I believe an honest mistake by Duke on their CDS (I informed them and they have since corrected it). I doubt USN would catch any of it. People working in magazine industry arenât known to be number savvy anyway and thereâs no way they can afford to hire an army to fact check given their slim profit margin.
Most of the stats submitted to USNews are also submitted to Moodyâs as one source of determining that schoolâs bond rating. So knowingly fudging these stats could have serious consequences, possibly criminal, far beyond the USNews rankings.
@TomSrOfBoston: What stats are submitted to Moodyâs?
Furthermore, if a college submits one set of numbers to Moodyâs and another to USNews, would either investigate? Who would know? I doubt Moodyâs releases the numbers they get.
And to echo @IWannaHelp, it may only take fudging a few stats, not all of them, to affect rankings.
@PurpleTitan This is an excerpt from Moodyâs website re: Northeastern University:
I saw the attachment to the Moodyâs submission (which I canât find now) that contained about 5 years of USNews statsâŠand they matched.
OK, so pretty much none of the
âFaculty resourcesâ category (âFaculty compensationâ, âPercent faculty with terminal degree in their fieldâ, âPercent faculty that is full timeâ, âStudent-faculty ratioâ, âClass sizeâ) would be reported to Moodyâs.
I could certainly see some gaming/fudging done there. Even the numbers reported to Moodyâs, even though accurate, could be gamed.
The new USNWR rankings come out in less than two weeks, so I think that the authors did the publishers a great favor by focusing the publicâs attention on college rankings. :!!
As @Wilson98 pointed out above, Northwestern is simply whining. It is mind-boggling to see a university that is ranked in the top-20 out of 2000+ colleges/universities in the country is talking about unfairness in the ranking system. How about the President of NW speaks to high schoolers or students at the universities that are ranked above Northwestern and asks where they would place Northwestern. And, asks why. Here is some data (Source: Parchment):
Between Dartmouth & Northwestern: 81% chose Dartmouth
Between Duke and Northwestern: 88% chose Duke
Between Brown and Northwestern: 75% chose Brown
I think the colleges could categorize the myriad of reasons for which a student leaves the college. If a student transfers they can state a reason for transfer. If a student doesnât enroll and they have had poor grades. Etc.
@fivesages: Why canât a prez of a highly-ranked college point out flaws in a ranking system?
Canât someone rich point out that there is a lot of wealth inequality in the US?
And why do you consider it âwhiningâ?
Whatâs amazing to me is just how emotionally attached some people are to some ranking system. They respond viscerally to criticism of a ranking system. That actually is more mind-boggling to me.
@PurpleTitan Anybody can point out the flaws in any system. But for a college president to spend his/her time on one of the many college rankings, done by a private institution, is not the best way to go about it.
@fivesages: Why not?
US News rankings have ruined college admissions. Schools are now fighting to gain however minuscule advantage they can just to move up a spot and playing all sorts of games to do so. Does it make any sense to proclaim one prestigious university âbetterâ than another? Thatâs why I especially appreciate Reed Collegeâs attitude about the whole thing, which is basically âScrew it. Weâre not playingâ.