How accurate is ChatGPT in predicting acceptances?

I’ve uploaded my resume, school profile, major, application round, residency, and grades, giving me an output that seems very generous. I mostly expected the reach/match/likelies but for the private reaches and selective out-of-state public schools, they seem to be generous. For example, NYU Stern is 20-30% RD, Tufts 30-40% RD.

I’ve seen people try using ChatGpt to predict if they’ll get it on here a while ago…how is it?

1 Like

About as accurate as the results of a Chance Me thread. IOW, not accurate. At all.

16 Likes

Going to try to put more info through GPT and Perplexity. Updating this thread with my actual results vs. what it said once I get some decisions!

1 Like

It cannot possibly generate a correct prediction, detailed historical applicant data is not available. Plus holistic admission is highly subjective and some of the decision variables are unknown.

3 Likes

Yup, I figured. I saw people doing it around here towards the end of last spring so I was curious.

It does acknowledge the limitations of “institutional priorities” and other factors. Also updates towards more realistic outcomes every time I provide new information, which I find interesting. Unfortunately, it’s hard to share the historical data because I don’t have anything beyond the regular unpaid versions :slight_smile:

Reportedly it didn’t do a good job for my S24 and his friends. Overall it was apparently too optimistic.

3 Likes

It would likely do a great job if it had excellent data sources in the model, which it doesn’t…
I think the model will have a harder time with super selective schools as scores and description will be solid for most applicants, but they still have to winnow down a lot…

That said, I put in my kids info for fun, and I am pretty impressed…nothing seems way off base and there seems to be some nuance - changes percentages based on major choices and alignment with schools and regions. 25’s lowest acceptance rate school is like 45% though…

w/o a big study, it is actually hard to know how good or not it is, of course. Just because your friend gets in and it says only 20%, that doesn’t make it wrong…

2 Likes

Okay - I don’t have RESULTS --but I typed in my daughter’s list and it seemed overly positive at a potential reach – it said 18% for Washington and Lee. I think it’s MAYBE 10% at MOST.

It was decent for targets – but I think it was a little pessimistic? It put Lafayette and Bucknell at 35% and I would put them in target rather than hard target. It put Union, Mount Holyoke, Brandeis, Dickinson in target range (around 50%). It put some of the likelies like Beloit and Miami University at 70% and I think her chances are higher at those. It put the Canadian schools at about 70% --and I don’t know realistically where she falls in those -but I think she’s a strong candidate for the schools we are putting on her list (Maple League NOT McGill, U of T etc). I would have put them a little higher (although 70% is obviously still better than not chancing).

My summary – it puts them on a correct scale from more likely to less likely -but the numbers themselves are perhaps a bit too optimistic on the reach side, and too pessimistic on the likely side.

2 Likes

Were they mostly optimistic on the reach schools? Or on all of them?

Also -wanted to say - I think these are way off 30-40% for Tufts? I think that’s a high number chance for even a very strong applicant? And 20-30% for NYU also seems very optimistic. Did you put in your targets and safeties?

For the highly selectives that practice holistic admissions, a big factor that AI will never know is the quality of your essays and LoR’s, and that is what is going to separate the “yes’s” from the final “no’s” and “maybe’s”. I guess you could put in a self evaluation as an assumption, but that’s pretty worthless. Parchment had a “chances” tool we used, but that really just set up the reach, match and safety categories. For kicks, I ran assumptions at the best possible, including hooks, and for HYPS type schools, I couldn’t get more than 18% – and this was when the admit rates for those schools was between 5-10%, not sub 5%.

2 Likes

Put in my profile and stats just for fun

In-State and Texas Options:

  1. Texas A&M University (TAMU): 80-90% (I’m auto-admit and have already gotten in)
  2. Southern Methodist University (SMU): 75-85%
  3. University of Texas at Austin (UT Austin): 50-65%

Public Out-of-State Options:

  1. University of Minnesota (UMinn): 70-80% (already accepted with honors)
  2. University of Georgia (UGA): 70-80%
  3. University of Alabama (UA): 90-95% (already accepted)

Selective Private Colleges and Universities:

  1. Georgetown University: 15-25%
  2. Colby College: 20-30%
  3. Grinnell College: 35-45%
  4. Drake University: 85-95% (Already accepted)
  5. Trinity University (TX): 75-85%
  6. Northeastern University: 30-40% (???)
  7. University of Richmond: 35-45%
  8. Wesleyan University: 20-30%
  9. Macalester College: 30-40%
  10. Gettysburg College: 70-80%
  11. Denison University: 70-80%
  12. Washington University in St. Louis (WashU): 15-25%
  13. Washington and Lee University (W&L): 20-30%
  14. Oberlin College: 35-45%
  15. Emory University: 20-30%
  16. Brown University: 5-10%

…It’s better than CollegeVine, for sure, but not great. Maybe a little optimistic about hard reaches like Emory and Northeastern, and a little pessimistic about safeties and targets. I feel like it might be trained on data that’s 5-10 years old.

Agreed!

I did put my safeties. They’re generous (UConn 90+% - makes sense, I’m in-state & UConn accepts most of my grade, SUNY Binghamton takes anyone with above a 3.0, etc.).

After pushing both models many times, Tufts fell to 25-35% (Economics RD) … likely deferred/waitlisted and NYU Stern (Economics) 15-20% …likely declined (Perplexity)

AND Tufts University (RD) – Economics Major: 30-40%, New York University (RD) – Economics Major: 20-30% (ChatGPT)

That’s a good question.

I think the kids I knew about were mostly trying it out for Reaches. I know my S24 was doing that. So the impression it was too optimistic plausibly may have reflected what they were doing.

1 Like

Perplexity seems to be more accurate since they also give the outcome rather than reach/match/etc. (and probably scalp more data from the internet to be honest).

Looking at school history, I would get into UMD (which Perplexity puts at 70-80% chance) while ChatGPT thinks is 50%-60% chance…we’ll see!

More significantly, ChatGPT thinks McCombs at UT Austin has a 40-50% chance while Perplexity thinks 50-60%, both of which are generous in my opinion. Both, I believe, struggle with opaque geographical nuances.

Let us know how it goes! I’ll post my final results along w/ my profile vs. what GPT says.

Super curious because you have a higher gpa than me but the one overlap school we have has a significantly different %.

Did you put your ECs/resume?

1 Like

What’s the overlap school?

yes, I just copied and pasted my resume

1 Like

I realized there are 2 actually (UT Austin & Emory), but don’t worry, I’m OOS for Austin and a completely different major for both schools and background I think, so there’s no way I’m hurting your chances lol.

No yeah I saw your post about McCombs; you’re good. I was just wondering what was the contrast in chances vs. stats you mentioned?

Emory is 40-50%. It could be location-related since my region isn’t heavily represented in the school. Genuinely no clue tbh.

1 Like