elsijfdl

this is one of those debates where no one is really going to listen to what anyone else says, but i strongly, strongly disagree with the assertion that peer assessment measures "pure academic strengths"

peer assessment is a manipulable, arbitrary conferral of generalized impression based on the vagaries of hearsay and speculation within the industry

and, as already noted, has everything to do with research output and prominent graduate faculty, factors which would be hard-pressed to have less to do with undergraduate experience.

xiggi

Oh well, this is a link for Barrons:

http://www.usnews.com/usnews/edu/col...rank_brief.php

hawkette

If you really think that research is the key to education and should determine how a college is ranked/graded, you might want to check with some non-academics to see if this view stands up.

IMO, academics often display a highly insulated (and inflated) view of the role that research plays in the educational experience and the world beyond. The reality is more likely that college research for a few technical areas can be quite important but, broadly speaking, its importance to the large majority of students is close to zero. And for alumni and employers, particularly in non-technical fields, the usefulness is pretty darn low.

As for the PA measure itself, who the heck knows what it measures. It means different things to different people, including those doing the grading, and almost certainly contains many uninformed, if not even malicious, opinions. PA is, without a doubt, the most corrupt and deleterious aspect of the USNWR survey.

kk19131

I love how 5 is "distinguished", and yet no school has such a score. I really want to know what's on those surveys.

barrons

I guess different links for different folks.

http://www.usnews.com/usnews/edu/col...ight_brief.php

Undergraduate ranking criteria and weights
The U.S. News college rankings, published on usnews.com Aug. 18, 2006, are based on several key measures of quality, described below. U.S. News uses these measures to capture the various dimensions of academic quality at each college. These measures fall into seven broad categories: peer assessment; graduation and retention rate; faculty resources (for example, class size); student selectivity (for example, average admission test scores of incoming students); financial resources; alumni giving; and, only for national universities and liberal arts colleges, graduation rate performance. The indicators include both input measures, which reflect the quality of students, faculty, and other resources used in education, and outcome measures, which capture the results of the education an individual receives.

Scores for each measure are weighted as shown to arrive at a final overall score. For a more detailed explanation of the ranking indicators and methods, please read our methodology and our definitions of ranking criteria, below.

Ranking Category Category Weight Subfactor Subfactor Weight
National Universities
and
Liberal Arts Colleges Universities (Master's)
and
Comprehensive Colleges (Bachelor's) National Universities
and
Liberal Arts Colleges Universities (Master's)
and
Comprehensive Colleges (Bachelor's)
Peer assessment 25% 25% Peer assessment survey 100% 100%
Student selectivity (Fall 2005 entering class) 15% 15% Acceptance rate 10% 10%
High school class standing—top 10% 40% 0%
High school class standing—top 25% 0% 40%
SAT/ACT scores 50% 50%
Faculty resources (2005) 20% 20% Faculty compensation 35% 35%
Percent faculty with top terminal degree 15% 15%
Percent full-time faculty 5% 5%
Student/faculty ratio 5% 5%
Class size, 1-19 students 30% 30%
Class size, 50+ students 10% 10%
Graduation and retention rate 20% 25% Average graduation rate 80% 80%
Average freshman retention rate 20% 20%
Financial resources 10% 10% Average educational expenditures per student 100% 100%
Alumni giving 5% 5% Average alumni giving rate 100% 100%
Graduation rate performance 5% 0% Graduation rate performance 100% 0%
Total 100% 100% — 100% 100%










Weights for national universities and liberal arts colleges





This graph shows the relative weights assigned to each category of indicator for national universities and liberal arts colleges.

Weights for universities-master's and comprehensive colleges-bachelor's





This chart shows the weights assigned to factors used to rank the universities-master's and comprehensive colleges-bachelor's. Because graduation rate performance is not used to rank these groups, the graduation and retention rate variables receive a higher weight.

Definitions of Ranking Criteria

Acceptance rate. The ratio of the number of students admitted to the number of applicants for the fall 2005 admission. The acceptance rate is equal to the total number of students admitted divided by the total number of applicants. Both the applications and acceptances only counted first-time, first-year students.

Alumni giving. The average percent of undergraduate alumni of record who donated money to the college or university. Alumni of record are former full- or part-time students that received an undergraduate degree and for whom the college or university has a current address. Graduates who earned only a graduate degree are excluded. Undergraduate alumni donors are alumni with undergraduate degrees from an institution that made one or more gifts for either current operations or capital expenses during the specified academic year. The alumni giving rate is calculated by dividing the number of appropriate donors during a given academic year by the number of appropriate alumni of record for that year. These rates were averaged for the 2004 and 2005 academic years. The percent of alumni giving serves as a proxy for how satisfied students are with the school.

Average freshman retention rate. The percentage of first-year freshmen who returned to the same college or university the following fall, averaged over the first-year classes entering between 2001 and 2004.

Average graduation rate. The percentage of freshmen who graduated within a six-year period, averaged over the classes entering between 1996 and 1999. (Note: This excludes students who transferred into the school.)

Class size, 1-19 students. The percentage of undergraduate classes, excluding class subsections, with fewer than 20 students enrolled during the fall of 2005.

Class size, 50+ students. The percentage of undergraduate classes, excluding class subsections, with 50 students or more enrolled during the fall of 2005.

Expenditures per student. Financial resources are measured by the average spending per full-time equivalent students on instruction, research, public service, academic support, student services, institutional support, and operations and maintenance (for public institutions only) during the 2004 and 2005 fiscal years. The number of full-time equivalent students is equal to the number of full-time students plus one-third of the number of part-time students. (Note: This includes both undergraduate and graduate students.) We first scaled the public service and research values by the percentage of full-time equivalent undergraduate students attending the school. Next, we added in total instruction, academic support, student services, institutional support, and operations and maintenance (for public institutions only) and then divided by the number of full-time equivalent students. After calculating this value, we applied a logarithmic transformation to the spending per full-time equivalent student, prior to standardizing the value. This calculation process was done for all schools.

Faculty compensation. The average faculty pay and benefits are adjusted for regional differences in cost of living. This includes full-time assistant, associate, and full professors. The values are taken for the 2004-2005 and 2005-2006 academic years and then averaged. (The regional differences in cost of living are taken from indexes from Runzheimer International.)

Faculty with Ph.D.'s. or top terminal degree. The percentage of full-time faculty members with a doctorate or the highest degree possible in their field or specialty during the 2005-2006 academic year.

Graduation rate performance. The difference between the actual six-year graduation rate for students entering in the fall of 1999 and the predicted graduation rate. The predicted graduation rate is based upon characteristics of the entering class, as well as characteristics of the institution. If a school's actual graduation rate is higher than the predicted rate, then the school is enhancing achievement. This measure is only included in the rankings for schools in the National Universities and Liberal Arts Colleges categories.

High school class standing. The proportion of students enrolled for the fall 2005 academic year who graduated in the top 10 percent (for national universities and liberal arts colleges) or 25 percent (master's and comprehensive colleges) of their high school class.

Peer Assessment. How the school is regarded by administrators at peer institutions. A school's peer assessment score is determined by surveying the presidents, provosts, and deans of admissions (or equivalent positions) at institutions in the school's category. Each individual was asked to rate peer schools' undergraduate academic programs on a scale from 1 (marginal) to 5 (distinguished). Those individuals who did not know enough about a school to evaluate it fairly were asked to mark "don't know." A school's score is the average score of all the respondents who rated it. Responses of "don't know" counted neither for nor against a school. The survey was conducted in the spring of 2006, and about 58 percent of those surveyed responded.

Proportion of full-time faculty. The proportion of the 2005-2006 full-time equivalent faculty that is full time. The number of full-time equivalent faculty is equal to the number of full-time faculty plus one third of the number of part-time faculty. (Note: We do not include the following: faculty in preclinical and clinical medicine; administrative officers with titles such as dean of students, librarian, registrar, or coach, even though they may devote part of their time to classroom instruction and may have faculty status; undergraduate or graduate students who are teaching assistants or teaching fellows; faculty on leave without pay; or replacement faculty for those faculty members on sabbatical leave.) To calculate this percentage, the total full-time faculty is divided by the full-time equivalent faculty.

SAT/ACT scores. Average test scores on the SAT or ACT of all enrolled first-time, first-year students entering in 2005. Before being used as a ranking indicator, the scores are converted to the percentile of the national distribution corresponding to that school's scores.

Student/faculty ratio. The ratio of full-time-equivalent students to full-time-equivalent faculty during the fall of 2005, as reported by the school. Note: This excludes faculty and students of law, medical, business, and other stand-alone graduate or professional programs in which faculty teach virtually only graduate-level students. Faculty numbers also exclude graduate or undergraduate students who are teaching assistants.

xiggi

Yep, especially when the notion that "faculty dedication to teaching" does not further your typical line of reasoning about the superiority of research over ... educating!

Anyway, shouldn't recognizing an incomplete source or failing to find the most appropriate one be part of good ... research?

Oh well, to each his own.

xiggi

Robert Morse (in his Morse Code) also stated that "In terms of the peer assessment survey, we at U.S. News firmly believe the survey has significant value because it allows us to measure the "intangibles" of a college that we can't measure through statistical data. Plus, the reputation of a school can help get that all-important first job and plays a key part in which grad school someone will be able to get into. The peer survey is by nature subjective, but the technique of asking industry leaders to rate their competitors is a commonly accepted practice. The results from the peer survey also can act to level the playing field between private and public colleges."

What is interesting is that "Plus, the reputation of a school can help get that all-important first job and plays a key part in which grad school someone will be able to get into." is a very different element from what USNews actually measures. Discussing, let alone measuring, the future value or marketability of the degrees earned at the undergraduate level shines by its absence in the Best Colleges publication.

Now, is there any doubt about what "The results from the peer survey also can act to level the playing field between private and public colleges" really mean and WHY it represents 25% of the final numbers. In other words, when empirical data fails to yield the EXPECTED results, let's make sure we keep a category that will get us there!

Mr. Morse might come to bitterly regret his decision of using the internet to explain the positions of USNews in a platform that people might read and actually spend the time deciphering the Codespeak!

barrons

I do not think my source was incomplete at all as it was actually much more detailed than yours. The authors of yours may have been writing somewhat off the cuff. Mine was the more definitive source IMHO.

Everyone protests the PA but nobody really has any evidence it is WRONG. Just whining about how it does not measure effectiveness. Well, when somebody figures out how to measure effectiveness--holding all other variables out--that will be an accomplishment.

I don't think research effectiveness automatically means poor teaching. I happen to think it improves teaching. Do you have any factual proof to the contrary or just those unfounded opinions that top researchers cannot be good teachers?

xiggi

Sure, Barrons!

barrons

Absolutely. You bet on the wrong horse.

elsijfdl

no, but there is always the fact that at most top schools, the researchers aren't teaching, and thus it matters little to undergrads whether or not their school developed a new kind of velcro.

http://www.marco-learningsystems.com...profchap5.html
http://www.getuponline.org/casualiza..._chronicle.htm
http://www.yalealumnimagazine.com/is...9_07/GESO.html

penn: 40% of classes taught by full-time professors
yale: 30% taught by full-time professors

hawkette

barrons,
You already know that I think that the PA is of very little value to most students and has no place in the college rankings system, but I want to give you a chance to defend your position. How would you suggest that a student looking for a college use the PA scores if he/she is not interested in the technical fields and/or performing research while in college?

norcalguy

The top schools on the peer assessment list look pretty good (this is probably how the world would rank the US schools):

1. Harvard University 4.9
1. Massachusetts Institute of Technology 4.9
1. Princeton University 4.9
1. Stanford University 4.9
1. Yale University 4.9

6. California Institute of Technology 4.7
6. University of California-Berkeley 4.7
6. University of Chicago 4.7

9. Columbia University 4.6
9. Cornell University 4.6
9. Johns Hopkins University 4.6

12. Duke University 4.5
12. University of Michigan-Ann Arbor 4.5
12. University of Pennsylvania 4.5

hawkette

Norcalguy,

Who is "the world?"

xiggi

Nope, Barrons! One needs of modicum of Critical Reading aptitude to be able to recognize the "right" horse or the ... use of sarcasm.