U.S. News Anticipates Higher Year-Over-Year Changes in Upcoming Rankings

They know who impresses them - not who is best. It’s a who is most popular metric and allows for little, if any, change.

USNWR has 4 graduation rate metrics for a total of 37% or 42% of overall score, depending on whether on high enough % submitted test scores.

  • 16% or 21% – 6-year graduation rate
  • 10% – graduation rate performance (predicted graduation rate / actual graduation rate)
  • 5.5% – 6-year graduation rate among Pell recipients
  • 5.5% – Pell graduation rate performance (Pell graduation rate / non-Pell graduation rate)

Ivy+ type colleges dominate the direct 6-year graduation rates. The performance based ones are less clear, as the specific weightings in the formula are unclear, and I don’t care to spend $60 to see the numbers.

1 Like

What metric is that? USNWR doesn’t directly measure “academic quality”, but they includes some measures that are probably correlated such as financial resources per student, faculty salary, and admin “marginal” / “distinguished” survey. If you are referring to the latter, note that the admins are rating other colleges, rather than the academic quality of their own. This is problematic on many levels. I expect few are experts on the academic quality of the 200+ other colleges on the list, so many just go by general feel, which is often influenced by the other colleges’ current USNWR ranking. Some may try to sabotage competitor’s rankings.

1 Like

The peer assessment rating also tends to be recursive. Highly ranked schools attract higher PA ratings from survey participants, schools that receive higher PA ratings rank more highly, highly ranked schools attract higher PA ratings from survey participants . . .

1 Like

This is the by major I’m talking about. Not overall.

I imagine profs keep up on publishing, awards, and talk to one another from time to time about who’s doing what cutting-edge thing, curricular content, lecture formats that work (and don’t), etc. That would be hard to quantify, but it must lead to some knowledge about which schools are doing what.

4 Likes

And they attend conferences, symposia, have grad students under their supervision who attended undergrad elsewhere, collaborate on research, are asked to review books and papers from colleagues at other institutions, sit in a courtroom where they are an expert witness and hear the testimony of the experts from the other side, AND try to replicate the results of a colleagues lab work before a journal will consider it “peer reviewed”.

5 Likes

I am not even an academic, but I know which universities (really which researchers) are top of the field in which I work, confidently… I know in which sub fields certain schools excel or have niche programs (at least for topics I have worked in)…

You glean it VERY quickly from grant award notifications, keeping up with literature, being asked to peer review (I actually do peer review for a couple journals), going to events, etc.

Now can I say that makes University A better place to go as an undergad? nope! That means University A has a better researchers who mainly worry about writing grants and their grad students and/or (depending on field) commercializing their research or partnering with industry. I have worked professional top .01% academics with absolutely top academics (think H-index in top 100 (on earth) and their interaction (and their research groups) with undergrads is approaching 0.

6 Likes

Undergraduate teaching professors aren’t the ones filling out the USNWR survey forms. The forms are sent to the president and dean of admissions, with the implication that they should be the ones filling it out. However, I expect that the admin support filling out the rest of the 100+ page USNWR forms often also end up filling out this section.

It sounds like you are describing which universities are leading in graduate research. This isn’t what they are supposed to be rating (for undergraduate USNWR ranking). They are instead instructed to rate the academic quality of the undergraduate programs on a scale of 5 = “distinguished” / 1 = “marginal”.

There are 400+ colleges in the discussed national universities categories. For how many of them do you think the admin filling out the form is an expert in the details of the other college’s undergraduate academic program? For example, suppose #183 ranked Drake made some dramatic changes to their undergraduate academic quality including making major changes to academic curriculum, degree requirements, adding additional academic support for students, … What percentage of admins at other colleges filling out the form are likely to recognize these changes and give Drake a higher score?

2 Likes

I suspect there is some transfer of new research/teaching/knowledge/enhancements to undergrads, especially in upper-level courses within a given major. (call it knowledge-seep… for lack of a better term.)

IE, there’s not an impregnable wall separating the star research profs, cutting them off from sharing their exploits with undergrads per se. At the very least, I figure they share with profs who do spend their time with undergrads, and maybe they share some of this new info with the kids. Maybe not in survey/intro courses, but more likely with upper-classmen in upper-level, major-specific courses.

I’m assuming here that no undergrads are allowed to get in on the research; if they can, all the better.

1 Like

As a footnote to this, it has been permissible, at least in the past, for administrators to rate their home institutions.

1 Like

With a consideration of earnings, the LACs currently tied at #13 may approach this relative order in the new rankings:

  1. Hamilton
  2. Middlebury
  3. Barnard
  4. Davidson
  5. Vassar
  6. Wesleyan
  7. Smith
  8. Grinnell

Irrespective of their order, a spread of 11 or so places across these schools — such as from 10th to 21st — may be seen with the new rankings. Some ties remain likely.

So wait. I’m confused. Where has USNews been getting all its post-graduate earnings information until now?

College Scorecard remains the source for earnings outcomes. The rankings last year evaluated “grads earning more than a high school grad” through this source. In the upcoming version, earnings with be considered with greater gradation, will be controlled for major and may differ in weighting from the previous 5%.

3 Likes

Well, Wesleyan consistently gets under-averaged (if that’s a concept) due to the large number of students going into film, theater and other performing arts (~11% of its graduates according to the Gordon Career Center.) A refocus on major to major, apples to apples, post-graduate earnings would be interesting.

3 Likes

In principle, this approach could raise Wesleyan performance by this metric. However, College Scorecard does not reveal major-specific earnings information for less common majors. Even for a popular major like economics, standardized data does not appear to be available for Wesleyan:

Earnings for Economics Majors for Eight School Reference Example, Four Years Out from Graduation

  1. Hamilton $130,633
  2. Middlebury $125,751
  3. Barnard $111,909
  4. Grinnell $110,724
  5. Vassar $102,584
  6. Davidson $101,310
  7. Smith $88,067

Wesleyan: N/A

(Source: College Scorecard).

So, wait. Scorecard already has this data (to a limited extent?)

1 Like

Yes. For liberal arts colleges, earnings by major information may be available for majors like economics, math and political science & government.

Here is College Scorecard for Wesleyan:

If you search by major in fields of study, you will see median earnings for students with that major. I am not sure why the majors with the highest N at Wesleyan are not showing earnings data. It seems like the N would be high enough in those majors, even among students who received any type of federal financial aid. College Scorecard only includes earnings info from students who received at least one type of federal financial aid.

1 Like

Note that the article says the new earnings by major metric is replacing the graduate indebtedness metric. It does not imply they are replacing the earning more than high school grad metric. This suggests the following change:

Old Weighting:

  • % Fed Aid Grads Earning More than HS Grads (>95% = perfect score) – 5%
  • Median Debt Among Fed Aid Grads – 5%
  • Other – 90%

New Weighting:

  • % Fed Aid Grads Earning More than HS Grads (>95% = perfect score) – 5%
  • Median Earnings by Major Among Fed Aid Grads – 5%
  • Other – 90%

This suggests the colleges that would most benefit from the change are ones that do better on median earnings by major than they do median debt. Last year, USNWR calculated median debt among fed aid recipients by only considering those who took out debt, so all colleges are >0. Among the listed LACs, the colleges that did best and worst in this metric are below. Max score is 100. These are estimates. There isn’t a huge fluctuation. All of these selective LACs score ~90/100.

  • Middlebury – 95
  • Hamilton – 91
  • Wesleyan – 91
  • Grinnell – 90
  • Smith – 89
  • Barnard – 89
  • Vassar – 87
  • Davidson – 87

The details of how USNWR will score colleges in the new earnings by metric is unclear. For example, what do they do for LACs when earnings can only be compared for ~1 major. Do they just compare earnings for economics and no other majors? Do they assume when earnings info is not available, students in that major have the median earnings for the full college? Do they not attempt to use the new metric at all for LAC ranking due to frequent lack of earnings?

Depending on how it is implemented, Smith stands to drop in rankings. As noted in Merc81’s post above, Smith tends to significantly lag other colleges of comparable USNWR ranking/selectivity on earnings. Being an all women college with relatively fewer students choosing to go in to fields like finance and consulting contributes. The Time Magazine rankings were mentioned earlier in this thread. In the Time Magazine rankings, Smith ranked last among the 487 ranked colleges in their student outcomes category, which is primarily based on how actual College Scorecard earnings (similar source to USNWR) compared to expected earnings.

4 Likes