CSM: College presidents plan 'U.S. News' rankings boycott

[tokenadult]

<p>

Given ND’s location in South Bend, Indiana it is remarkable that they draw the caliber of students they do.

</p>

<p>The guy I know from Minnesota told me, I’m pretty sure after he visited both campuses, that he preferred the feel of South Bend, IN to that of Cambridge, MA. I have very close family ties to rural areas (my mother grew up in farm country, with the nearest neighbor a quarter-mile away). But I like urban environments, and my preferences, having visited both of those towns, would differ in preferring Cambridge to South Bend.</p>

[mathmom]

<p>There are lots and lots of Catholics at our high school, but over the last few years only four applied to Notre Dame and none got in. Fordham is probably the most popular of the well known Catholic colleges that kids from our neck of the woods apply to. Villanova and Boston College also get quite a few applications, and with considerably lower rankings, the College of New Rochelle and Iona College.</p>

[siserune]

<p><a href=“%5Bb%5Dtokenadult%5B/b%5D%20wrote:”>quote</a> Well, if you have any disagreements with the revealed preferences study, why don’t you meet the authors and discuss how to improve it?

[/quote]
</p>

<p>The question asked of you (or anyone else impressed by the study) was what makes the RP ranking better than instantly available large-scale data, such as the ratio I defined above, or numbers extractible from the annual National Merit report. Glib referral to the authors sends a clear signal that the study is being trumpeted by people who don’t understand its content. We see in this thread, for example, people who actually think that a 75 percent victory rate for Harvard over MIT was a finding of the study as published in the New York Times, when actually it is a complicated summary statistic with no direct interpretation in terms of cross admit battles. </p>

<p>Since you were in touch with the authors, did you ask them for the data? Why isn’t it available, when any layman can immediately understand and interpret the “revealed preferences” directly from the cross-admit tables, something that isn’t true of the rankings.</p>

<p>

The study speaks for itself about its methodology, the data set used, and how it compares with other approaches to ranking colleges.

</p>

<p>The study does not do any of the above – have you read it? It does not disclose the data it is based on, i.e. the table of cross-admit results. It does not address known methodological problems (instability) in their model caused by lack of cross-admit data as one goes down the rank list. It does not indicate why it is better (be it less manipulable, easier to interpret, more informative, etc) than simple metrics such as the one I suggested, or a simple tabulation of the cross-admit results. That’s for starters. </p>

<p>

Go ahead and publish a paper if you think you have some new insight about college comparisons that hasn’t been adequately explored in the existing literature.

</p>

<p>Sounds like bluffing. Is it too much to ask that before posting another dozen links to this study, the Harvard cheerleaders bother to read and understand what it says?</p>

[siserune]

<p>

Regarding Revealed Preference, you can decide for yourselves whether or not to “drink the kool-aid”. When I looked I noticed several anomolies, such as prominence of BYU & Notre Dame. I believe the methodology is flawed because non-applicants have themselves expressed a relative dis-preference for a school by not applying to it, and this is not captured.

</p>

<p>That’s right, though I think the special-interest schools like BYU, ND, BC get ranked high primarily because of indirect victories (they “beat the school that beat the school”, as in boxing rankings) and only secondarily because of self-selection per se. There is a huge anomaly when they rank Caltech second in the country based on between 7 and 12 matriculation victories; Caltech is a great school but nobody believes that it is generally preferred to Yale and Princeton or that it is more popular among engineers than MIT. </p>

<p>The false-positive comes about because Caltech has the strongest applicants, whose other options are better than the alternatives of applicants to any other school, and when they select Caltech it is in preference to MIT, Harvard and Stanford (which carries more weight than beating a no-name) and the minority of battles that Caltech loses also weigh less because they aren’t to low-ranked schools. The latter benefit is partly due to sample size: a larger number of Caltech battles might have revealed losses to Carnegie Mellon, RPI, Georgia Tech, Cooper Union, and the full-tuition honors program at low-ranked state schools.</p>

[asteriskea]

<p>The Top American Research Universities according to The Center for Measuring University Performance. </p>

<p>

Who’s number one? The quintessential American question. We all want to stand first in line, first in the hearts of our country, first in the polls, first in the standings. The pursuit of Number One is surely an important thing in sports, but for universities, being first is not as important as being among the best…

</p>

<p>The Center for Measuring University Performance was founded by its co-editor John Lombardi, Chancellor of the University of Massachusetts Amherst.</p>

<p><a href=“http://mup.asu.edu/research.html[/url]”>http://mup.asu.edu/research.html&lt;/a&gt;&lt;/p&gt;

<p><a href=“http://mup.asu.edu/research_data.html[/url]”>http://mup.asu.edu/research_data.html&lt;/a&gt;&lt;/p&gt;

<p>Advisory Board:
<a href=“http://mup.asu.edu/advisor.html[/url]”>http://mup.asu.edu/advisor.html&lt;/a&gt;&lt;/p&gt;

[tokenadult]

<p>

@asteriskea

The Top American Research Universities according to The Center for Measuring University Performance.</p>

<p>“Who’s number one? The quintessential American question. We all want to stand first in line, first in the hearts of our country, first in the polls, first in the standings. The pursuit of Number One is surely an important thing in sports, but for universities, being first is not as important as being among the best. . . .” </p>

<p>The Center for Measuring University Performance was founded by its co-editor John Lombardi, Chancellor of the University of Massachusetts Amherst.

</p>

<p>It’s interesting how many approaches there are to this “who’s number one” question, which has resulted in lively participation in this thread. I look at this issue mostly through the lens of the young people I know through a statewide parents association, and consider what colleges those young people might find most suitable. There certainly is diversity of opinion on the issue of how colleges differ from one another. </p>

<p>Do you know, off-hand, of any journals that publish articles about these issues? What learned back-and-forth have you seen about the many issues raised in this thread in formal scholarly publications?</p>

[MarathonMan88]

<p>The report linked in asteriskea’s post #39 has some good articles about exactly these issues.</p>

[asteriskea]

<p>The Center for Measuring University Performance also provides the following link:
<a href=“http://mup.asu.edu/publications.html[/url]”>http://mup.asu.edu/publications.html&lt;/a&gt;&lt;/p&gt;

<p>Then you have journals such as the Journal of Consumer Culture:</p>

<p><a href=“http://joc.sagepub.com/cgi/content/abstract/5/3/338[/url]”>http://joc.sagepub.com/cgi/content/abstract/5/3/338&lt;/a&gt;&lt;/p&gt;

<p>The journal “Research in Higher Education” runs articles such as “Measuring Quality: A Comparison of U.S. News Rankings and NSSE Benchmarks”</p>

<p>The Center of Inquiry in the Liberal Arts offers the following:</p>

<p><a href=“http://liberalarts.wabash.edu/assessment[/url]”>http://liberalarts.wabash.edu/assessment&lt;/a&gt;
<a href=“http://liberalarts.wabash.edu/cila/home.cfm?news_id=3630[/url]”>http://liberalarts.wabash.edu/cila/home.cfm?news_id=3630&lt;/a&gt;&lt;/p&gt;

<p>For those so inclined, here is a link to the abstract "Measuring Quality: A Comparison of U. S. News Rankings and NSSE Benchmarks " There are almost four pages of cited references, that includes, of course, works by Pascarella and Kuh.</p>

<p><a href=“http://209.85.135.104/search?q=cache:2wbrKE2Sj78J:nsse.iub.edu/pdf/research_papers/Pike_Measuring_Quality.pdf+Measuring+Quality:+A+Comparison+of+U.S.+News+Rankings+and+NSSE+Benchmarks&hl=en&ct=clnk&cd=2&client=firefox-a[/url]”>http://209.85.135.104/search?q=cache:2wbrKE2Sj78J:nsse.iub.edu/pdf/research_papers/Pike_Measuring_Quality.pdf+Measuring+Quality:+A+Comparison+of+U.S.+News+Rankings+and+NSSE+Benchmarks&hl=en&ct=clnk&cd=2&client=firefox-a&lt;/a&gt;&lt;/p&gt;

[cadbury]

<p>asterikea, thanks for all the great resources!</p>

[barrons]

<p>This site has links to many of the major rankings–one stop shopping</p>

<p><a href=“http://www.library.uiuc.edu/edx/rankings.htm[/url]”>http://www.library.uiuc.edu/edx/rankings.htm&lt;/a&gt;&lt;/p&gt;

[asteriskea]

<p>There is no dearth of material on this subject. From the journal, Higher Education in Europe, Vol. XXVII, No. 4, 2002 “Some Guidelines for Academic Quality Rankings” by Marguerite Clarke is also worth perusing.</p>

<p><a href=“http://209.85.135.104/search?q=cache:6znR9PE3Tg4J:www.griffith.edu.au/conference/university-rankings/pdf/clarke-guidelines-quality-ranking.pdf+guidelines+for+Academic+quality+rankings+clarke&hl=en&ct=clnk&cd=1&client=firefox-a[/url]”>http://209.85.135.104/search?q=cache:6znR9PE3Tg4J:www.griffith.edu.au/conference/university-rankings/pdf/clarke-guidelines-quality-ranking.pdf+guidelines+for+Academic+quality+rankings+clarke&hl=en&ct=clnk&cd=1&client=firefox-a&lt;/a&gt;&lt;/p&gt;

[MarathonMan88]

<p>Thanks for all of the great links! Here’s one more. It lists a lot of resources on the assessment, outcome-based measurement side of the ledger. </p>

<p><a href=“http://www.colby.edu/administration_cs/ir/assessment/index.cfm[/url]”>http://www.colby.edu/administration_cs/ir/assessment/index.cfm&lt;/a&gt;&lt;/p&gt;

[asteriskea]

<p>Thanks for the link to the Colby site. </p>

<p>Article by Alexander McCormick, director of the Carnegie Foundation, “First, Do No Harm” on accountability proposals is also worth the read:</p>

<p>

…it’s not sufficient to simply select a set of performance measures. It’s equally important to consider how the system will affect behavior. A well-designed accountability system motivates substantive change, not merely gaming the system. And the last thing you want is a system that undermines useful diagnostic tools in the name of accountability…

</p>

<p><a href=“Page not found | Carnegie Foundation for the Advancement of Teaching”>Page not found | Carnegie Foundation for the Advancement of Teaching;

[siserune]

<p>

“…non-applicants have themselves expressed a preference by not applying.” </p>

<p>Umm…yeah. So why does this make the methodology flawed?

</p>

<p>Because there is a lot of additional information in the application patterns that would correct the picture from matriculation battles alone. For one thing, the anomalous advantage to niche schools would be diluted or disappear.</p>

[siserune]

<p><a href=“marite%20wrote:”>quote</a> Are the authors claiming that revealed preferences are a superior way of ranking colleges

[/quote]
</p>

<p>Yes. They claim that their method is a superior way of producing the preference rating used as an input to USNews-style rankings. They say that use of their method or one like it would alleviate pressure on colleges to manipulate rankings. They also claim (despite a disclaimer at the end of the paper) that such rankings reflect market-aggregated information about the actual quality of schools, not just a popularity contest. At several points in the article they “sell” their approach as an improvement over the status quo.</p>

<p>

The one piece of information that stuck in my mind was the alleged strategy of Princeton of lowering the rate of admission of students scoring between the 94 and 98% percentile on the SAT.

</p>

<p>The empirical part of the study is a gold mine of data that I hope they will publish. The graphs of admission rate by SAT score are priceless. </p>

<p>

The authors also acknowledged the limitation of their methodology: the schools had to be of similar reputation and to be all need-based, so that there was more likelihood of students comparing apples and apples.

</p>

<p>You may be thinking of comments made in CC about the study, because no such qualifications are made in the paper itself. It would destroy the rationale for their method if it were valid mainly for the handful of need-based schools of similar reputation. In fact their method relies on indirect comparisons of dissimilar schools (chains of “A beats B who beats C”) precisely because there is a lack of direct cross-admit data for most pairs of schools.</p>

[asteriskea]

<p>More than interesting update article: according to Inside Higher Ed, the numbers are growing. Apart from Moravian, other colleges signing on include: the College of the Southwest; Colorado, Eckerd, McDaniel, Northwestern (Minn.), Philander Smith, Shimer, Unity, and Washington & Jefferson Colleges, Denison, Furman, Missouri Baptist, Naropa and Ohio Wesleyan Universities. </p>

<p>As I wrote in several previous posts, it is also significant that at this point there is nothing close to a consensus among those university officials who haven’t joined the boycott yet. Many are clearly sympathetic to the protest against the PA survey but are opting to take a wait and see position - these IHEs want to see the development of viable alternative measures to provide students and parents with a replacement for the rankings before they make a move against USNWR. Pressure is mounting nonetheless as the boycott movement makes headway - since there is also the question of just how many colleges will decide to sign on in the wake of the upcoming Annapolis meeting.</p>

<p>For all the details:
<a href=“http://www.insidehighered.com/news/2007/05/18/usnews[/url]”>http://www.insidehighered.com/news/2007/05/18/usnews&lt;/a&gt;&lt;/p&gt;

[MarathonMan88]

<p>Even if you’ve already read the article linked in the post above, it’s worth a return trip, as there have been forthright comments added by Ed Hershey (one assumes the Ed Hershey who’s been pr guy at Cornell, Colby, Reed) and Chris Nelson, the president of St. John’s, MD.</p>

[xiggi]

<p>

Pressure is mounting nonetheless as the boycott movement makes headway - since there is also the question of just how many colleges will decide to sign on in the wake of the upcoming Annapolis meeting.

</p>

<p>Considering there are about 4,000 colleges in the country, the “participation” bar has been raised from 0.03% to 0.75%.</p>

[xiggi]

<p>

Significantly, one of the new colleges joining the effort is Philander Smith College, whose president has enlisted the National Association for Equal Opportunity in Higher Education to send out an appeal urging all historically black colleges to join the campaign. The request going to black colleges will argue that the U.S. News rankings are inherently unfair to historically black colleges.

</p>

<p>With all due respect to Philander Smith College, could someone explain the extent of the “unfairness” of USNews towards this Arkansas school? Isn’t this school helped by merely being listed in the first place, even if it is in the category of “least selective schools?”</p>

<p>How does US News truly interfere with the mission of P. Smith and its President? Does the USNews influence the 80% admission rate, the average 15 ACT score, and the 4% 4 year graduation rate? </p>

<p>Forgive me for being cynical, but the joining of P. Smith is not significant and hardly relevant. It is about one thing: the same issue that compelled schools such as Reed and Drew to join the battle, and that is called FREE PUBLICITY.</p>

[StickerShock]

<p>I agree with xiggi about the free publicity. I had never heard of Reed before finding CC. It’s really not on the radar in my area, despite by all accounts its fine reputation. I only know of Drew because it’s 20 miles from my home & is know as a good school where merit aid is awarded. It certainly has no national name. And no disrespect intended, but how many people have heard of Philander Smith?</p>