OpenAI simply can’t be trusted on any benchmarks: https://news.ycombinator.com/item?id=42761648
The International Math Olympiad isn’t an AI benchmark.
It’s an annual human competition.
OpenAI claims gold-medal performance at IMO 2025
161–170 of 737 posts
Re: OpenAI claims gold-medal performance at IMO 2025
#162From that thread: "The model solved P1 through P5; it did not produce a solution for P6." It's interesting that it didn't solve the problem that was by far the hardest for humans too. China, the #1 team got only 21/42 points on it. In most other teams nobody solved it.
In the IMO, the idea is that the first day you get P1, P2 and P3, and the second day you get P4, P5 and P6. Usually, ordered by difficulty, they are P1, P4, P2, P5, P3, P6. So, usually P1 is "easy" and P6 is very hard. At least that is the intended order, but sometime reality disagree. Edit: Fixed P4 -> P3. Thanks.
In this case P6 was unusually hard and P3 was unusually easy https://sugaku.net/content/imo-2025-problems/
Re: OpenAI claims gold-medal performance at IMO 2025
#163Earlier quoted context omitted.
I mean these problems you can get better with practice. But if you haven't solved many before and can do them after an afternoon of thought I would be very impressed. Not that I don't believe you, it's just in my experience people like this are very rare. (Also I assume they have to have some degree of familarity of some common tricks otherwise they would have to derive basic number theory from scratch etc and that s…
I think honestly it's probably different experiences and skillsets. I find these sort of things doable bar dumb mistakes by myself, yet there will be other things I'll get stressed and not be able to do for ages (some lab skills no matter the number of times I do them and some physical equation derivations that I regularly muck up). I maybe sometimes assume that what comes easy for me, comes easy for all, and what I…
Here you go — you did this type of problems as a kid/teenager. 1) you likely have a talent for it 2) you have some training.
I did participate in math/informatics olympiads as a teenager and even taught it a little and from my experience, some type of people just _like_ that sort of problems naturally, they tickle their minds, and given time this people would develop to insane levels at it.
'Normal people', in my experience, even in math departments, don't like that type of problems, and would not fare well with them.
Re: OpenAI claims gold-medal performance at IMO 2025
#164The cynicism/denial on HN about AI is exhausting. Half the comments are some weird form of explaining away the ever increasing performance of these models I've been reading this website for probably 15 years, its never been this bad. many threads are completely unreadable, all the actual educated takes are on X, its almost like there was a talent drain
The overconfidence/short sightedness on HN about AI is exhausting. Half the comments are some weird form of explaining how developers will be obsolete in five years and how close we are to AGI.
I don’t typically find this to be true. There is a definite cynicism on HN especially when it comes to OpenAI. You already know what you will see. Low quality garbage of “I remember when OpenAI was open”, “remember when they used to publish research”, “sama cannot be trusted”, it’s an endless barrage of garbage.
Re: OpenAI claims gold-medal performance at IMO 2025
#165Re: OpenAI claims gold-medal performance at IMO 2025
#166Earlier quoted context omitted.
The overconfidence/short sightedness on HN about AI is exhausting. Half the comments are some weird form of explaining how developers will be obsolete in five years and how close we are to AGI.
I don’t typically find this to be true. There is a definite cynicism on HN especially when it comes to OpenAI. You already know what you will see. Low quality garbage of “I remember when OpenAI was open”, “remember when they used to publish research”, “sama cannot be trusted”, it’s an endless barrage of garbage.
its honestly ruining this website, you cant even read the comments sections anymore
Re: OpenAI claims gold-medal performance at IMO 2025
#167[flagged]
As far as I can tell, OpenAI didn't participate, and isn't claiming they participated. Note the fairly precise phrasing of "gold medal-level performance": they claim to have shown performance sufficient for a gold, not that they won one.
Re: OpenAI claims gold-medal performance at IMO 2025
#168Earlier quoted context omitted.
A third party tried this experiment with publicly available models. OpenAI did half as well as Gemini, and none of the models even got bronze. https://matharena.ai/imo/
I feel you're misunderstanding something. That's not "this exact experiment". Matharena is testing publicly available models against the IMO problem set. OpenAI was announcing the results of a new, unpublished model, on that problems set. It is totally fair to discount OpenAI's statement until we have way more details about their setup, and maybe even until there is some level of public access to the model. But you'r…
Fair enough, edited.
Re: OpenAI claims gold-medal performance at IMO 2025
#169Earlier quoted context omitted.
Or some can spot a euphoric bubble when they see it with lots of participants who have over-invested in 90% of these so called AI startups that are not frontier labs.
What does this have to do with the math Olympiad? Why would it frame your view of the accomplishment?
Why don’t they release some info beyond a vague twitter hype post? I’m beginning to hate OpenAI for releasing statements like this that invariably end up being less impressive than they make it sound initially.
Re: OpenAI claims gold-medal performance at IMO 2025
#170The cynicism/denial on HN about AI is exhausting. Half the comments are some weird form of explaining away the ever increasing performance of these models I've been reading this website for probably 15 years, its never been this bad. many threads are completely unreadable, all the actual educated takes are on X, its almost like there was a talent drain
Probably because both sides have strong vested interests and it’s next to impossible to find a dispassionate point of view. The Pro AI crowd, VC, tech CEOs etc have strong incentive to claim humans are obsolete. Many tech employees see threats to their jobs and want to poopoo any way AI could be useful or competitive.
That's just another way to state that everybody is almost always self-serving when it comes to anything.