Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI claims gold-medal performance at IMO 2025

twitter.com

Post reply on HN
Print view View original

591–600 of 737 posts

Re: OpenAI claims gold-medal performance at IMO 2025

#591

Post by danlitt »

mewpmewp2 wrote: post #462
danlitt wrote: post #365

Earlier quoted context omitted.

No, you are right, this hyper-numericalism is just astrology for nerds.

In military they estimate distances this way if they don't have proper tools. Each says a min max range and then where there's most overlap, that will be taken. It's a reasonable way to make quick intuition based decisions when no other way is available.

If you actually try to flesh out the reasoning behind the distance estimation strategy it will turn out 100x more convincing than the analogous argument for bayesian probability estimates. (and for any bayesians reading this, please don't multiply the probability by 100)

Re: OpenAI claims gold-medal performance at IMO 2025

#592

Post by suddenlybananas »

lexandstuff wrote: post #553
chairhairair wrote: post #77

OpenAI simply can’t be trusted on any benchmarks: https://news.ycombinator.com/item?id=42761648

I don't fault you for maintaining a healthy scepticism, but per the President of the IMO: "It is very exciting to see progress in the mathematical capabilities of AI models, but we would like to be clear that the IMO cannot validate the methods, including the amount of compute used or whether there was any human involvement, or whether the results can be reproduced. What we can say is that correct mathematical proofs…

Is b) really that unlikely?

Re: OpenAI claims gold-medal performance at IMO 2025

#593

Post by demirbey05 »

davis wrote: post #585
demirbey05 wrote: post #4

Progress is astounding. Recently report published about evaluation of LLMs on IMO 2025. o3 high didn't even get bronze. https://matharena.ai/imo/ Waiting for Terry Tao's thoughts, but these kind of things are good use of AI. We need to make science progress faster rather than disrupting our economy without being ready.

Here they are: https://mathstodon.xyz/@tao/114881419368778558

Appreciated

> I will not be commenting on any self-reported AI competition performance results for which the methodology was not disclosed in advance of the competition.

Re: OpenAI claims gold-medal performance at IMO 2025

#594

Post by amelius »

sorokod wrote: post #338

Not even bronze. https://news.ycombinator.com/item?id=44615695

From the article:

> Btw, we are releasing GPT-5 soon, and we’re excited for you to try it. But just to be clear: the IMO gold LLM is an experimental research model. We don’t plan to release anything with this level of math capability for several months.

Re: OpenAI claims gold-medal performance at IMO 2025

#595

Post by lanza »

fnordpiglet wrote: post #95
johnecheck wrote: post #48

Wow. That's an impressive result, but how did they do it? Wei references scaling up test-time compute, so I have to assume they threw a boatload of money at this. I've heard talk of running models in parallel and comparing results - if OpenAI ran this 10000 times in parallel and cherry-picked the best one, this is a lot less exciting. If this is legit, then we need to know what tools were used and how the model used…

Why is that less exciting? A machine competing in an unconstrained natural language difficult math contest and coming out on top by any means is breath taking science fiction a few years ago - now it’s not exciting? Regardless of the tools for verification or even solvers - why is the goal post moving so fast? There is no bonus for “purity of essence” and using only neural networks. We live in an era where it’s hard…

Because the usefulness of an AI model is reliably solving a problem, not being able to solve a problem given 10,000 tries.

Claude Code is still only a mildly useful tool because it's horrific beyond a certain breadth of scope. If I asked it to solve the same problem 10,000 times I'm sure I'd get a great answer to significantly more difficult problems, but that doesn't help me as I'm not capable of scaling myself to checking 10,000 answers.

Re: OpenAI claims gold-medal performance at IMO 2025

#597

Post by zug_zug »

causal wrote: post #118
zug_zug wrote: post #63

Earlier quoted context omitted.

I feel like I've noticed you you making the same comment 12 places in this thread -- incorrectly misrepresenting the difficulty of this tournament and ultimately it comes across as a bitter ex. Here's an example problem 5: Let a1,a2,…,an be distinct positive integers and let M=max⁡1≤i Find the maximum number of pairs (i,j) with 1≤i<j≤n for which (ai +aj )(aj −ai )=M.

Where did you get this? Don't see it on the 2025 problem set and now I wanna see if I have the right answer

I asked chatGPT. However it's saying that's 2022 problem 5, however that seems to be clearly wrong... Moreover I can't find that problem anywhere so I don't know if it's a hallucination or something from it's training set that isn't on the internet....

Re: OpenAI claims gold-medal performance at IMO 2025

#598

Post by ctoth »

torginus wrote: post #577
modeless wrote: post #117

Noam Brown: > this isn’t an IMO-specific model. It’s a reasoning LLM that incorporates new experimental general-purpose techniques. > it’s also more efficient [than o1 or o3] with its thinking. And there’s a lot of room to push the test-time compute and efficiency further. > As fast as recent AI progress has been, I fully expect the trend to continue. Importantly, I think we’re close to AI substantially contributing…

Thing is, for example, all of classical physics can be derived from Newton's laws, Maxwell's equations and the laws of Thermodynamics, all of which can be written on a slip of paper. A sufficiently brilliant and determined human can invent or explain everything armed only with this knowledge. There's no need to train him on a huge corpus of text, like they do with ChatGPT. Not sure what this model's like, but I'm qui…

And the billions of years of evolution and the language that you use to explain the task to him and and the schooling he needs to understand what you're saying it and... and and and?

Re: OpenAI claims gold-medal performance at IMO 2025

#599

Post by archon1410 »

tmule wrote: post #584
archon1410 wrote: post #477

Earlier quoted context omitted.

It is not "supremacist" to believe that depriving hundreds of millions of people from higher education in their native language is deeply unjust. This reflection was prompted by a comment on why Indian languages are not represented in international competitions, which was prompted by a comment on the competition being available in many languages. Discussions online have a tendency to go off into tangents like this. I…

> self-loathing elites in India Your disdain for English-speaking Indian elites (pejoratively referred to as ‘Macaulayites’ by Modi’s supporters) is quite telling. That said, as I mentioned earlier, this kind of discourse doesn’t belong here.

My disdain is for the fact that hundreds of millions of Indians cannot access higher education in their native language, and instead of simply learning a foreign language as a subject like the rest of world, they have the bear the burden[1] of learning things in a foreign language which they have to simultaneously learn. I have disdain for the people responsible for this mess. I do not have any disdain for any language-speaking class, specially not one which I might be part of.

[1]https://www.mdpi.com/2071-1050/14/4/2168

Re: OpenAI claims gold-medal performance at IMO 2025

#600

Post by SonOfLilit »

torginus wrote: post #573
dylanbyte wrote: post #16

These are high school level only in the sense of assumed background knowledge, they are extremely difficult. Professional mathematicians would not get this level of performance, unless they have a background in IMO themselves. This doesn’t mean that the model is better than them in math, just that mathematicians specialize in extending the frontier of math. The answers are not in the training data. This is not a mode…

From my vague rememberance of doing data science years ago, it's very hard not to leak the training set. Basically how you do RL is that you make a set of training examples of input-output pairs, and set aside a smaller validation set, which you never train on, to check if your model's doing well. What you do is you tweak the architecture and the training set until it does well on the validation set. By doing so, you…

The questions were published a few days ago. The 2025 IMO just ended.
Post reply on HN