OpenAI claims gold-medal performance at IMO 2025

Post by mathteddybear » Sat, Jul 19, 2025, 8:06 PM UTC

keeda wrote: post #367

parsimo2010 wrote: post #68

Earlier quoted context omitted.

I am a professor in a math department (I teach statistics but there is a good complement of actual math PhDs) and there are only about 10% who care about these types of problems and definitely less than half who could get gold on an IMO test even if they didn’t care. They are all outstanding mathematicians, but the IMO type questions are not something that mathematicians can universally solve without preparation. The…

> They are all outstanding mathematicians, but the IMO type questions are not something that mathematicians can universally solve without preparation. So IMO is basically the leetcode of Mathematics.

Post by c1ccccc1 » Sat, Jul 19, 2025, 8:08 PM UTC

lucianbr wrote: post #488

c1ccccc1 wrote: post #407

Earlier quoted context omitted.

You should basically assume they are pulled from thin air. (Or more precisely, from the brain and world model of the people making the prediction.) The point of giving such estimates is mostly an exercise in getting better at understanding the world, and a way to keep yourself honest by making predictions in advance. If someone else consistently gives higher probabilities to events that ended up happening than you di…

Is there some database where you can see predictions of different people and the results? Or are we supposed to rely on them keeping track and keeping themselves honest? Because that is not something humans do generally, and I have no reason to trust any of these 'rationalists'. This sounds like a circular argument. You started explaining why them giving percentage predictions should make them more trustworthy, but w…

Post by hyghjiyhu » Sat, Jul 19, 2025, 8:09 PM UTC

emp17344 wrote: post #346

falcor84 wrote: post #276

Earlier quoted context omitted.

I don't get it, how do you "big data cheat" an AI into solving previously unencountered problems? Wouldn't that just be engineering?

I mean, solutions for the 2025 IMO problems are already available on the internet. How can we be sure these are “unencountered” problems?

Post by lucianbr » Sat, Jul 19, 2025, 8:11 PM UTC

diggan wrote: post #86

johnecheck wrote: post #74

Earlier quoted context omitted.

The key bit here is whether the LLM doing the cherry picking had knowledge of the solution. If it didn't, this is a meaningful result. That's why I'd like more info, but I fear OpenAI is going to try to keep things under wraps.

> If it didn't We kind of have to assume it didn't right? Otherwise bragging about the results makes zero sense and would be outright misleading.

Post by tunesmith » Sat, Jul 19, 2025, 8:13 PM UTC

sigmoid10 wrote: post #73

z7 wrote: post #9

Some previous predictions: In 2021 Paul Christiano wrote he would update from 30% to "50% chance of hard takeoff" if we saw an IMO gold by 2025. He thought there was an 8% chance of this happening. Eliezer Yudkowsky said "at least 16%". Source: https://www.lesswrong.com/posts/sWLLdG6DWJEy3CH7n/imo-challe...

While I usually enjoy seeing these discussions, I think they are really pushing the usefulness of bayesian statistics. If one dude says the chance for an outcome is 8% and another says it's 16% and the outcome does occur, they were both pretty wrong, even though it might seem like the one who guessed a few % higher might have had a better belief system. Now if one of them had said 90% while the other said 8% or 16%,…