Brown is too easy

[siserune]

<p>

Doesn’t MyChances use samples of sometimes as few as 3 students?

</p>

<p>I had the same question, but from the confidence interval they reported it looks like the sample was 25-35 direct cross admit battles between Brown and Columbia. The total sample size is larger than the Revealed Preference study but a little thinner for the top schools and with data of lower quality. The site should definitely report the number of battles for each pair of schools, in addition to the confidence interval.</p>

<p>

Also, doesn’t the data come from a single counselor who could easily direct students down certain paths of his/her own preference?

</p>

<p>It’s the same as College Confidential, self-reported data from thousands of users who submit stats profiles and admissions results, but with an Elo point cross-admit ranking similar to Revealed Preferences, and a display of the raw cross-admit rates (with confidence intervals). I think the site is run by a medical student with a computer/stats hobby, not an admissions counselor. </p>

<p>

However, as an aggregate, Brown still well out performs the number 8 slot, and I’d love to see some analysis on the likelihood that Brown scored that much higher totally by chance. Confidence in very basic statistics is calculated that way, and even if Brown may curiously lose in a few head-to-heads with lower ranked schools, the aggregate still may be accurate and elevated through far greater than predicted success against other schools.

</p>

<p>That’s exactly what the “confidence probabilities” table in the article claims to quantify. They assign 79 percent confidence (in the sense of a confidence interval) to the Brown > Columbia ranking output. This is not a 79 percent cross-admit victory rate. It means that they performed thousands of simulations in which the rankings were re-computed and in 79 percent of those Brown ranked higher than Columbia. They claim that these simulations are, under some assumptions, samples from the distribution they are trying to access and if so, really do represent confidence in the usual statistical sense (the likeliness of the result appearing by chance, given their model).</p>

<p>

If anything, this may suggest there are fewer cross-admits with Brown and some of the schools surrounding it, and that instead, more students who chose to go to Brown are self-selected.

</p>

<p>Brown is self-selected less than Caltech and MIT but more so than other Ivy League schools, so yes, this should bias its results upward (as an indicator of what all students want, not what students who chose to apply to Brown and other schools want) in the RP model. </p>

<p>

[analysis] Does that make sense? It’s far harder to describe in writing than I anticipated.

</p>

<p>You described it perfectly as far as I can tell. The “being unique” (Brown, Wellesley, Notre Dame, Caltech, …) makes the ranking complicated if it is understood as a desirability rating, because on the one hand it’s a plus that is attractive to many, on the other hand it leads to the upward self-selection bias. </p>

<p>

For example, no school I applied to was nearly as selective as Brown despite the fact that I had stats that made me competitive anywhere.

</p>

<p>This is part of what’s screwed up in the RP method. Such an applicant expresses a lower preference for HYPS and a higher preference for Brown but hurts the latter’s rating (and helps the former) compared to a more lukewarm preference where one does apply to those other schools, then turns them down for Brown.</p>

<p>

Regardless, this is all speculation helping to keep a nonsense thread bumped over nonsense splitting hairs.

</p>

<p>It is intellectually interesting to dissect the RP ratings but they should certainly not be viewed as in any way reliable or authoritative, even as measurements of their own 1999-2000 data.</p>