Emory Finds Intentional Misreporting of SAT/Rank Data, Revamps Procedures

Colleges & Universities Emory University
41

<p>Does anyone know what scores Emory reported to US News? If it’s the same as what it reported to the College Board, then the reported midpoint would be 2120 (I believe both use data for the year before last, i.e. fall 2010). The fall 2010 CDS from Emory puts the midpoint at 2045. If this is right, the midpoint in the most recent ranking is 75 points higher than it should be.</p>

<p>Berkeley (#21) may edge Emory out for #20 (same SAT midpoint, higher proportion in the top 10%, lower acceptance rate).</p>

42

<p>Why the frack anyone cares about the USNWR rankings (or any other) is beyond me. Even if every piece of data was reported with 100% accuracy, the rankings are MEANINGLESS. Why can’t people get that through their heads?</p>

<p>First of all, USNWR is trying to quantify an undefinable property. Best College? What does that even mean? It varies for almost every individual that is going to college.</p>

<p>Second, even if you grant some leeway on that point and say there is some kind of “generally accepted” concept of what makes some universities better than others, the parameters used by USNWR have never been shown to reflect that concept. The justification usually used is “Well, Harvard, Yale, etc. always show up at the top and everyone knows they are the best, so it must be right”. Any scientist or statistician will tell you that line of thinking is categorically false. Harvard, Yale and a few others are in relatively unique situations when it comes to issues such as percent acceptance, yields, historical brand recognition, etc. This may (or may not, but again, without strict definitions and testing procedures, who knows) mean that the measurement parameters for the rest of the institutions have to be adjusted to reflect these very real differences.</p>

<p>For example, peer assessment is a huge factor in the equation used by USNWR. Does anyone really think the people answering that question have as informed an opinion as to what is really going on at UC Davis or, for that matter, Emory as they (at least) think they do at Harvard and Yale? Probably they have no idea of the real goings on at these latter institutions either, but it is a beauty contest, so HYPS are safe choices to rate very highly. Not saying they are wrong, just that there is likely no basis for most responders to really have an informed opinion.</p>

<p>Third, the very fact that USNWR changes what parameters they use, the weightings given to them, and the fact that they can be manipulated not just by being dishonest but by taking certain other actions to bias the results shows how unscientific and unreliable this is. A small example is that a school can double or triple the number of apps it gets through mass mailings and other means, thus lower its percent acceptance rate. Now that isn’t a great example because that particular statistic is only 1.5% of the formula used by USNWR (although it probably weighs more heavily in the minds of potential applicants. A 25% acceptance rate sounds a lot more prestigious than a 50% or 60% acceptance rate). But if worrying about admissions and thus tangentially the rankings was your full time job, I am sure one could come up with other ways to push the needle in the right direction without actually lying.</p>

<p>Anyway, sorry to rant but the rankings are such total BS. Emory would be the same great school if it were ranked #10, #20, or #40. Psychologically it is really hard to swallow the worse ranking, that is just how we are wired. But logically it is absolutely true.</p>

<p>bernie12 - As an aside, when you have a really long post, break it up into shorter paragraphs, please. You have good things to say, but reading a run on paragraph makes it more difficult. Just a suggestion.</p>

43

<p>fallenchemist…thank you for your great post. Totally agree! Also agree with your bernie 12 comment. Bernie12 I like to read your posts, but difficult to read. Coming from the business world, you will lose your audience if you do not break up your long paragraphs. </p>

<p>Cannot wait till we can move on from this “mis reporting of data.”</p>

44

<p>

So says the guy whose alma mater fails to publish a CDS…</p>

45

<p>Cannot understand why Emory would do this when it resulted in a difference of just 30 or 40 points total re the medians? The only thing I can think of is they wanted to keep the medians at 1400 (CR plus M) and 2100 total. Yet, they could have achieved the same results by focusing more on SAT scores is admissions (ala Vandy). </p>

<p>Still kudos to Emory for self-reporting this problem.</p>

46

<p>muckdogs07 - I feel fairly confident it was indeed more about appearances than about the USNWR rankings. After all, they had to be able to roughly calculate that it made very little difference in their final position within the rankings. People can make a lot of snide comments about that ("well, if they are dumb enough to cheat they are probably too dumb to make those kinds of calculations), but there are numerous examples in everyday life in more traditional businesses where people fudge things just a little bit to get it just above or just below some level that is perceived as some “magical goal”. Kind of like pricing something at 4.99 cents instead of $5, but in reverse in this case.</p>

<p>Very silly, but very human. And still wrong, not making excuses for them at all. Just MHO on why they most likely did it, as opposed to some kind of reward, other than some kind of (false) satisfaction. Which is weird, since they knew they were fudging there shouldn’t be any satisfaction, but then people cheat in online games all the time too, and still feel good about “winning”. People can be strange.</p>

47

<p>People “can be” strange? People “are” strange lol. Think about it, they wouldn’t cheat if integrity or appearance of illegitimate while winning meant more to them than winning alone. Thanks for the suggestion. Just don’t really think of sentence structure and stuff when typing on CC, kind of just bombs away. Will take you readers into account when I feel like it.</p>

48

<p>Oxford’s stats for the incoming class: </p>

<p>[Oxford</a> College - 2012 Entering Class Profile](<a href=“http://oxford.emory.edu/admission/entering-class-profile/]Oxford”>http://oxford.emory.edu/admission/entering-class-profile/)</p>

<p>Slightly surprising…</p>

49

<p>No they aren’t surprising (I don’t think so at least). They had plenty more applications and they get more competitive every year. In addition, their stats. are usually very similar to UGA’s (now the top percentile SATs are higher though. Everything else is the same). Either way, good for them, they are finally making some progress in admissions (getting even more people that can handle their rigor w/some finesse).</p>

50

<p>What makes this difficult is that it happened for 12 years! While USN&WR feels the small diviation in stats would not really affect the rankings, they are careful not to discuss any punative effect. Iona College was found to have falsely reported their stats 2 years ago. it was done only for 2 years and they too thought it would not have a big affect on their ranking. </p>

<p>They went from 30th on the Liberal Arts ranking to Not Rated: a drop of some 30 points!</p>

<p>So I am curious what the final outcome will be once the “Good they came forward” is balanced against the “Bad they did it for 12 years” factors. Emory is so strong, they will come back in a year, but I don’t think USN&WR will only drop it a point or two.</p>

51

<p>

It is a mathematical formula. USN&WR uses no judgement on their part to assign the place in the rankings. This isn’t like the ESPN coaches poll, other than the approx. 25% factor of “peer assessment”. But that isn’t USNWR’s opinion, that is the opinion of people in academia, along with a small percentage for high school guidance counselors. So to the extent these people feel that this kind of incident causes them to rate Emory lower, it could affect their ranking. But again, that has nothing to do with any judgement by USNWR.</p>

<p>The rest is based on numerical data such as admission rate, test scores, class sizes, retention rates, etc. So it is subject to misreporting by the institutions (obviously), but again no judgement calls by USNWR.</p>

<p>So your comment, while I understand the thinking behind it, actually makes no sense. If Iona was Not Rated the year after they were found to be cheating, I suspect it had more to do with USNWR deciding they could not trust the data rather than the correct data being that different. After all, even if the data was much worse, they would still get a ranking, albeit a much lower one.</p>

52

<p>

</p>

<p>How so? Honestly, I’m surprised the admissions rate isn’t in the 30s.</p>

<p>Highlighted Changes:
Average GPA: 3.79 (Up from 3.55 last year)
ACT/SAT scores are up slightly; it looks like our 75th percentile roughly equates with 40th percentile at the College of Arts and Sciences–a significant improvement facilitated in part by the correction of data by CAS
42% Male vs. 58% Female
75% of Students from Outside of Georgia (major improvement)
58% Attended Public School (~7% Drop)</p>

53

<p>aigiqinf: Didn’t a slightly higher % attend public school at CAS? Is Oxford’s a tad higher due to the sample size (and it being more sensitive to say, 10 more freshmen than last year being from a private school)?</p>

54

<p>How do these threads get so off track? lol</p>

55

<p>You can only talk about the misinterpreting data issue but so much. There isn’t much to speculate about. It happened, we’re mad and disappointed, it may or may not change the rank and freak some people out, etc. That’s about as far as this issue gets.</p>

56

<p>Right. But if people want to talk about the details of the Oxford stats, they should start a new thread.</p>

57

<p>

</p>

<p>I disagree. Any effect of the false data from the College of Arts and Sciences is material to this discussion. Following the thread, part of the discussion moved from a discussion of how the corrected data would compare to the other divisions (especially Oxford). The stats were posted because comparison data (i.e., incoming students data) had just been posted and showed the the gap between the College of Arts and Sciences and Oxford has decreased.</p>

<p>Also, Oxford became a part of the conversation in part because President Wagner’s e-mail did not clearly differentiate that the problem was entirely within the College of Arts and Sciences, not in Emory’s other four divisions that grant undergraduate degrees.</p>

58

<p>But the effect of the false data was not involved at all in posts 48, 49, 52 and 53. Clearly post 48 could (and should) have been a new thread. Even arguing about it has nothing to do with the thread as titled. I was just trying to get the discussion to either go back to its stated purpose, or to have it end if, as Bernie states, it is all talked out. After all, if you were someone looking for a discussion about what Oxford’s stats are and how they are changing, would you click on this thread given its title? I wouldn’t have.</p>

59

<p>“So your comment, while I understand the thinking behind it, actually makes no sense. If Iona was Not Rated the year after they were found to be cheating, I suspect it had more to do with USNWR deciding they could not trust the data rather than the correct data being that different. After all, even if the data was much worse, they would still get a ranking, albeit a much lower one.”</p>

<p>I think you are seeing this situation rather myoptically, fallen chemist, looking at just numbers and not the the bigger picture. This is more than just some stats misreported.</p>

<p>Your comment that USN&WR “Decided” they could not trust Iona’s data as to why they were dropped so far, yet think Emory just used different data is hopeful and irrational: what Iona …or Clearmont…did they only did for 2 years, what Emory did they did for 12 years. USN&WR may not take kindly to incorrect information reported as it hurts their credibility as a leading college research tool and may “Decide” they cannot trust Emory’s data as they did not trust Iona’s. To do less would subject them to charges of favoratism or bias. While YOU may see Emory’s action as a small difference in data, many news articles and editorials I have read on this have used words such as “Fraud”, “Cheating” and “Lies”. If it were purely just a mathematical formula, Iona would not have fallen so far as you point out. The fact that it did suggests more is at play than just mathematics. Do you really believe as fine a school as Emory is, it should be above the valuations and judgements given to other schools when they report false data? …for 12 years? You may not think Emery cheated, but many do.</p>

<p>I guess we will have to wait untill next year to see what effect this does have on the ratings. Emory will bounce back.</p>

60

<p>Actually, I think USNWR does play favorites. It kind of designed the metrics and methodology behind certain values that the makers or its readers holds. And currently, those metrics favor a handful of private schools (or publics that may as well be privates given the way they are run) being at the very top. In addition, if they drop schools “as punishment for lying”, I actually think that should dent their credibility more. That’s like being an 8 year child throwing a hissy fit about their parents telling them there is no tooth fairy or something. If USNWR was mature, they would have in the Iona case and any other school’s case, simply used the updated data, plugged into their stupid formula and generate a number, and then see where the school lands in the rankings. Also, Emory did kind of data “switch”, so it had two sets of data which were accurate. The issue was they decided to use the one they were not supposed to when sending data to ranking agencies. At least the numbers were not completely made up. Regardless, Emory is wrong for this. It was so useless and unnecessary.</p>

<p>On a side note, I question the metric as a whole in that I kind of think “selectivity” should be describing how hard it is to get in as opposed to who yields (the admitted students’ data describes this). If they wanted enrolled data, they should have a special metric called “caliber” (perhaps they do), but rules are rules.</p>