2011 PA Scores

[BioDomer]

<p>Tufts, Wake, ND lose on the PA because they are very strong undergraduate institutions with mostly weak grad schools. Since there’s no clear direction to rank say undergraduate quality the grad programs are factored into the PA. Counselors send kids off for undergrad not grad school so they rank based on perceived undergrad strength.</p>

[xiggi]

<p>

Tufts, Wake, ND lose on the PA because they are very strong undergraduate institutions with mostly weak grad schools. Since there’s no clear direction to rank say undergraduate quality the grad programs are factored into the PA. Counselors send kids off for undergrad not grad school so they rank based on perceived undergrad strength.

</p>

<p>What does this say about the PA? </p>

<p>However, although the instructions of USNews are clearly lacking, the DO specifically ask to measure the quality of the UNDERGRADUATE. The problem remains that the responders pay little to attention to that CLEAR distinction, and this to the great joy and glee of the PA supporters who benefit from a PA that reflects both relevant and irrelevant intangibles. </p>

<p>As far as the counselors, I believe their source of information is last year’s edition of the USNews. This could explain why the range of the first 20-40 schools is much narrower than the PA.</p>

[UCBChemEGrad]

<p>

@xiggi

However, although the instructions of USNews are clearly lacking, the DO specifically ask to measure the quality of the UNDERGRADUATE. The problem remains that the responders pay little to attention to that CLEAR distinction, and this to the great joy and glee of the PA supporters who benefit from a PA that reflects both relevant and irrelevant intangibles.

Not exactly, xiggi. They specifically ask the user to rate a university’s academic programs from distinguished (5) to marginal (1). Colleges with higher PA scores (typically) have more renowned academic programs than colleges with lower scores.</p>

<p>They probably do take into consideration UNDERGRAD…For example, Berkeley has many distinguished academic programs and would probably be ranked 5…however, since the university is larger and has less resources to devote to all of its undergrads, its PA score is discounted. </p>

<p>As I’ve said numerous times before, not many colleges could be rated 5 (or distinguished), because if there were, colleges become undistinguished.</p>

<p>

The problem remains that the responders pay little to attention to that CLEAR distinction

No, the problem is the distinction is far from CLEAR when trying to separate a school’s graduate academic programs from its undergraduate academic programs.</p>

[xiggi]

<p>

Not exactly, xiggi. They specifically ask the user to rate a university’s academic programs from distinguished (5) to marginal (1). Colleges with higher PA scores (typically) have more renowned academic programs than colleges with lower scores.</p>

<p>They probably do take into consideration UNDERGRAD…For example, Berkeley has many distinguished academic programs and would probably be ranked 5…however, since the university is larger and has less resources to devote to all of its undergrads, its PA score is discounted.

</p>

<p>UCB, as I often do, I’ll tell you … do not take my word for it. Why don’t we look at the instructions for the LACs (because I just used them a few days ago) while I’ll locate the exact text of the survey for a national university. </p>

<p>

Here are the Directions for Overall Ratings:</p>

<ol>
<li><p>Please rate the academic quality of undergraduate programs at the following schools in the liberal arts colleges category. These schools are primarily undergraduate colleges that award fifty percent or more of their baccalaureate degrees in the liberal arts.</p></li>
<li><p>Please review the entire list first, considering each program’s scholarship record, curriculum, and quality of faculty and graduates.</p></li>
<li><p>Using a black pen, rate each school with which you are familiar on a scale from marginal (1) to distinguished (5) by marking an “X” in the corresponding box. If you are not familiar with a school’s faculty, programs, and graduates, please mark “don’t know.”

</p></li>
</ol>

[UCBChemEGrad]

<p>^ Uh huh… undergraduate academic programs.</p>

<p>IMO, LAC academic programs are more difficult to measure on distinction due to less visibility and the following impacts:

1. Less visibility due to less research (which rightly or wrongly is where visible academic achievements are made).
2. Less visibility tends to present the great LACs in a more homogenized light vs. their university counterparts.</p>

<p>

considering each program’s scholarship record, curriculum, and quality of faculty and graduates.

what factor do you think is most visible to administrators? I say faculty quality…therefore, this factor is likely the most significant in academics minds while filling out the survey.</p>

<p>Also, note the instructions state to consider a program’s graduates, NOT what their students happened to do on a 3 hour test taken during their junior year of high school.</p>

[xiggi]

<p>Ready for the National Universities?</p>

<p>Why not look at the infamous Wisconsin survey for the directions AND how the esteemed offiicial answered:</p>

<p><a href=“mailto:dailycal@dailycal.org_20090818_151146.pdf”>dailycal@dailycal.org_20090818_151146.pdf</a></p>

<p>Here are the differences in bold:</p>

<p>

Here are the Directions for Overall Ratings:</p>

<ol>
<li><p>Please rate the academic quality of undergraduate programs at the following schools in the **national universities category. These schools are major research universities and leading grantors of doctorate degrees. **</p></li>
<li><p>Please review the entire list first, considering each program’s scholarship record, curriculum, and quality of faculty and graduates.</p></li>
<li><p>Using a black pen, rate each school with which you are familiar on a scale from marginal (1) to distinguished (5) by marking an “X” in the corresponding box. If you are not familiar with a school’s faculty, programs, and graduates, please mark “don’t know.”

</p></li>
</ol>

<p>What did NOT change is this … Please rate the academic quality of undergraduate programs!</p>

[xiggi]

<p>UCB, we were talking about the instructions. And I answered your “Not exactly, xiggi. They specifically ask the user to rate a university’s academic programs from distinguished (5) to marginal (1).” </p>

<p>I think that it is clear that you had omitted the most important word, namely undergraduate. Again, do not take my word for it! :)</p>

<p>Anyhow, here’s the position of James Barker: </p>

<p>

In an interview Monday, he defended his approach. “The request from U.S. News is to measure the academic quality of undergraduate programs,” Barker said. “It did not say research programs, it did not say prestige. It did not say size of endowments, or anything other than undergraduate education. And I took that charge seriously, measuring what I would think would be the full package of the undergraduate experience,” including faculty-student ratio, relationships between faculty and students in and out of class … do they spend time having lunch together.</p>

<p>“I believe that Clemson does that better than anyone,” he said. “That’s why my ranking for Clemson is where it is. You’ll notice I did not give anyone a 5, because I did not believe any of us had reached that level… I’m a hard grader.”

</p>

<p>It is interesting that, on the one hand, you always want to convince everyone that those people are experts and have the best knowledge of the colleges, and that people like me are fools not to understand that, but on the other hand, when one of those experts speaks up … it is a different story. </p>

<p>Oh well!</p>

[UCBChemEGrad]

<p>^ Looking at one survey result is sorta meaningless…unless you just want to criticize that person’s opinion…which is more likely your motive. USNews could have thrown out some of its responses. You need to look at the aggregate results of over 2,000 surveys…the raw number surveyed washes out strategic voting.</p>

[UCBChemEGrad]

<p>BTW, your link doesn’t work…you’re emailing the Daily Cal…which is kinda odd. :)</p>

[UCBChemEGrad]

<p>

I think that it is clear that you had omitted the most important word, namely undergraduate. Again, do not take my word for it!

And I responded to you by saying this:

No, the problem is the distinction is far from CLEAR when trying to separate a school’s graduate academic programs from its undergraduate academic programs.

</p>

[xiggi]

<p>UCB, this is no fun! </p>

<p>The shenanigans of the Wisconsin and the Clemson officials are OLD news. I did write that I posted a link to the survey to look at both the questions and the answers provided by the bandit who answered it. Do I seek to criticize that person’s opinion? I think I am a bit late to that party!</p>

<p>The question you should ask yourself should be about the prevalence of this type of response. Again, this IS old news. The same news source that covered the survey debacle also ran a story about the results they saw in the 18 surveys they analyzed. If the result was positive is for you to find out! It is a google search away! </p>

<p>Does anything I typed today different from what I have written in the past? No, because those are the facts, or as they’d say in the woods, the bear facts! </p>

<p>At the end, believe what you want and dismiss what you do not like to hear. It won’t change anything, especially not my own conclusions.</p>

[UCBChemEGrad]

<p>Xiggi, </p>

<p>Compare these academic programs and tell me the differences:</p>

<ol>
<li>Harvard graduate economics vs. Harvard undergrad economics</li>
<li>Stanford graduate psychology vs. Stanford undergrad psychology</li>
<li>Notre Dame graduate engineering vs. Notre Dame undergraduate engineering</li>
<li>USC graduate film program vs. USC undergraduate film program</li>
<li>Duke graduate biomedical engineering vs. Duke undergraduate biomedical engineering</li>
<li>Penn Wharton MBA vs. Penn Wharton business undergraduate</li>
</ol>

[xiggi]

<p>Try this link</p>

<p><a href=“http://www.insidehighered.com/media/news_documents_and_files/2009/08/madisonform[/url]”>http://www.insidehighered.com/media/news_documents_and_files/2009/08/madisonform&lt;/a&gt;&lt;/p&gt;

[UCBChemEGrad]

<p>^ That link just opens my email browser…thinks its an email address.</p>

<p>OK, first link works.</p>

[UCBChemEGrad]

<p>

The shenanigans of the Wisconsin and the Clemson officials are OLD news.

Oh and of course I know it’s old news. But I’m not the one posting it. :)</p>

[xiggi]

<p>And you were the first to question my motive for posting it.</p>

[UCBChemEGrad]

<p><em>shrugs</em> I wasn’t really questioning…more reinforcing. I know your position.</p>

<p>Are you going to make an effort to address some of the examples in Post #52?</p>

[xiggi]

<p>

Are you going to make an effort to address some of the examples in Post #52?

</p>

<p>Not really, and for a number of reasons. </p>

<ol>
<li>I do not see the relevance to the Peer Assessment’s instructions</li>
<li>I do not know those programs</li>
<li>I DO know why you ask the question.</li>
</ol>

<p>Considering the three points listed above, I find it easier to pass. Except, for repeating that evaluating the graduate programs should be not be relevant to the responders, and if they, just as I did admit, do NOT know enough to evaluate the undergraduate programs, they SHOULD answer I DO NOT KNOW. They should not speculate or respond to a questions that is NOT posed. </p>

<p>As far as your position, I speculate that you will say that the departments are so intertwined that it is impossible to evaluate them individually. However, again, that goes back to my point that the CURRENT Peer Assessment is an incredibly poorly executed and futile exercise.</p>

[barrons]

<p>Xiggi leaves out the rest of the Wisconsin story. The official who did that was relieved of his duties in that regard. He also offers a more detailed explanation. Of course no private instutions were even subject to such scrutiny as they would never divilge such information. </p>

<p>[Ranking</a> rancor](<a href=“http://host.madison.com/ct/news/local/education/university/article_94c84a58-f8c6-53d3-a9db-598eebe961ea.html]Ranking”>http://host.madison.com/ct/news/local/education/university/article_94c84a58-f8c6-53d3-a9db-598eebe961ea.html)</p>

<p>With so much weight already given to the quality and selectivity of students, I think the time is well past due for the faculty to get its own ranking as part of the overall ranking. For research U’s it would be fairly easy to come up with a list of quantitative factors at least as compelling as SAT scores and class rank/gpa etc. Faculty awards, publications, memberships the various learned societies, research grants awarded etc. </p>

<p>For LACs I leave it to others to figure out a system as most LAC faculty have much thinner resumes.</p>

[xiggi]

<p>

Xiggi leaves out the rest of the Wisconsin story. The official who did that was relieved of his duties in that regard. He also offers a more detailed explanation. Of course no private instutions were even subject to such scrutiny as they would never divilge such information. </p>

<p>Ranking rancor

</p>

<p>The PA fans are a pretty funny bunch. Actually, since they tend to contradict themselves on several occasions, I am not sure if they are actually fans. After all, perhaps Barrons does not like the PA too much, but only likes the impact of the PA on the final scores. Or something equaly illogical.</p>

<p>In the meantime, I’d say that “leaving the rest” out the Wisconsin story should be pretty accurate since I did NOT attempt to tell the entire story, especially since it’s old news. </p>

<p>On the other hand, it’s pretty interesting that the parts of the story I left out might have been VERY much along the lines of what** I have written **about the PA, including in this thread.</p>

<p>Let’s have a look"</p>

<p>

One of the key components of the U.S. News and World Report rankings is a peer assessment survey, which accounts for 25 percent of a school’s point total. This survey asks respondents to “**rate the academic quality of undergraduate programs **at the following schools” on a five-point scale that ranges from “distinguished” to “marginal.” Survey takers are also directed to “review the entire list first, considering each program’s scholarship record, curriculum and quality of faculty and graduates.”</p>

<p>And one of those “apathetic respondents” was UW-Madison’s own Brower. He ranked 259 of the 262 schools as “adequate,” which is the second-lowest rating on the scale. He gave the highest marks, “distinguished,” to UW-Madison and the New School, based in New York City, and rated Arizona State as “marginal.”</p>

<p>Brower says he rated most schools as adequate because he wanted to respond in a “neutral way” to a “poorly designed survey” that he believes has gained far too much importance.</p>

<p>“The problem with an overall ranking of colleges is that without set criteria, you don’t know what it means,” says Brower, who argues that ratings of specific programs based on narrow criteria could, on the other hand, be “extremely helpful” to prospective students.</p>

<p>Brower filled out the survey that had been mailed to Julie Underwood, at the time UW-Madison’s interim provost. Underwood said she passed this task along to Brower because she felt he was the most qualified person within the provost’s office to fill out the survey. </p>

<p>Brower is not the only UW-Madison official to raise questions about the U.S. News & World Report rankings. Former UW-Madison chancellor John Wiley told the Cap Times one year ago that these ratings “have no meaning.”</p>

<p>“It’s like astrology,” Wiley said in a candid conversation just days before Biddy Martin took over as chancellor in September 2008. “It’s amusing. And people read it, so it’s in the paper. But that’s about it. The idea that there is a best university in the country is scientific nonsense.”</p>

<p>**Without a more specific list of criteria, **Brower believes the survey portion of the rankings is akin to asking “‘What’s a better car, a Prius or a Jaguar?’ Well, if you’re talking about gas mileage, it’s the Prius. But if you’re talking about luxury, it’s the Jaguar.”</p>

<p>Brower insists he wasn’t trying to game the system to help UW-Madison in the rankings.</p>

<p>“I had no reason to think that my ratings would have any impact,” he said. “UW-Madison already is high in the rankings.”

</p>

<p>In fact, had I posted the “untold” story, I probably would have been pillored and accused of selective posting. </p>

<p>Where there a few items worth sharing? Facts that corroborate my often repeated claims that people who sign the survey rarely fill it in, and that whoever does it should reply I DO NOT KNOW to about 95% of the evaluation. </p>

<p>

<a href=“http://www.insidehighered.com/news/2009/08/19/rankings[/url]”>http://www.insidehighered.com/news/2009/08/19/rankings&lt;/a&gt;
This news surprised Julie Underwood, then-interim provost, when informed by Inside Higher Ed this month. That’s because she didn’t fill out the survey submitted in her name. As many other officials do, she sent it to an administrator to complete on her behalf – in this case, Aaron Brower, vice provost for teaching and learning, who had never filled it out before. Underwood did not give her input or approval before he returned it to U.S. News. </p>

<p>Asked why he didn’t check “don’t know” for all the other institutions, or refuse to complete the survey, Brower says the possibility never occurred to him: “It seemed like this was a task that I was to do and I was to do it the best I could.” And he insists that he wasn’t trying to game the system, noting that his survey was one of hundreds.

</p>

<p>As I said, the PA fans are a pretty funny bunch! And they make me smile quite often.</p>