Businessweeks undergraduate rankings

Dominar
Thu, 28 Mar 2013 12:16:30 +0000

I’m no statistics expert, but having followed these rankings for a few years now, I have to wonder at their methodology. Unusually large swings from year to year and weird results suggest that certain factors carry too much weight and depend too heavily on response rate. I mean, how can a school’s “teaching quality” swing a full letter grade (or more) in one year unless all of the good professors from the prior year suddenly quit? How can a school’s “employer ranking” jump 13 places in one year?

I inquired at one school S is considering about its DISMAL recruiter survey scores, because its starting salary, placement services and student satisfaction numbers were all very high (i.e., students seemed happy with the school’s recruiting/placement efforts). The placement director told me that the survey rewards schools who do the best job of getting their on-campus recruiters to respond to the survey, and that they were trying to educate their employers about the importance of responding. Apparently, a non-response is treated as a poor grade. Also, BW apparently “unskews” otherwise favorable employer numbers for those schools whose programs aren’t familiar to large number of employers. So, schools that are recruited more locally because of their distance to larger urban centers will see their positive numbers “adjusted” downwards.

If you look at their methodology, the “student satisfaction” score counts for 30% (!) of the ranking, while the “recruiter/employer survey” score counts for 20%. That’s just crazy. Moreover, the “facilities,” “placement” and “teaching quality” grades are also based entirely on student surveys – again, very subjective and reflects more a school’s ability to motivate its students to respond positively to the survey.

The few objective measures included in the rankings, i.e., academic quality (GPA, class size, student/teacher ratio), starting salaries and MBA feeder school ranking, count for only 50%. And even they artificially reward schools closer to large urban areas (higher salaries, more students pursuing MBAs) and private schools with more money to spend on professors (class size, ratios). The “academic quality” measure also has at least one subjective element, i.e., the self-reported # of hours students spend on schoolwork as some sort of proxy for “quality.”

All in all, I find the information contained in the survey interesting, but it’s important to know where it comes from and how it’s used in the overall ranking rather than just assume, for example, that the University of Richmond is really #17 overall in the country (LOL!). Not that UR is a bad school, but really – does anyone believe it had the same “academic quality” as Wharton, Berkeley and Carnegie Mellon in 2012?