Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI claims gold-medal performance at IMO 2025

twitter.com

Post reply on HN
Print view View original

711–720 of 737 posts

Re: OpenAI claims gold-medal performance at IMO 2025

#711

Post by lostmsu »

torginus wrote: post #577
modeless wrote: post #117

Noam Brown: > this isn’t an IMO-specific model. It’s a reasoning LLM that incorporates new experimental general-purpose techniques. > it’s also more efficient [than o1 or o3] with its thinking. And there’s a lot of room to push the test-time compute and efficiency further. > As fast as recent AI progress has been, I fully expect the trend to continue. Importantly, I think we’re close to AI substantially contributing…

Thing is, for example, all of classical physics can be derived from Newton's laws, Maxwell's equations and the laws of Thermodynamics, all of which can be written on a slip of paper. A sufficiently brilliant and determined human can invent or explain everything armed only with this knowledge. There's no need to train him on a huge corpus of text, like they do with ChatGPT. Not sure what this model's like, but I'm qui…

Right, humans are pretrained on terabytes of sensory data instead.

Re: OpenAI claims gold-medal performance at IMO 2025

#712

Post by gallerdude »

beering wrote: post #633
skepticATX wrote: post #354

Earlier quoted context omitted.

OpenAI’s systems haven’t been pure language models since the o models though, right? Their RL approach may very well still generalize, but it’s not just a big pre-trained model that is one-shotting these problems. The key difference is that they claim to have not used any verifiers.

What do you mean by “pure language model”? The reasoning step is still just the LLM spitting out tokens and this was confirmed by Deepseek replicating the o models. There’s not also a proof verifier or something similar running alongside it according to the openai researchers. If you mean pure as in there’s not additional training beyond the pretraining, I don’t think any model has been pure since gpt-3.5.

Local models you can get just the pretrained versions of, no RLHF. IIRC both Llama and Gemma make them available.

Re: OpenAI claims gold-medal performance at IMO 2025

#713

Post by meroes »

Workaccount2 wrote: post #631
meroes wrote: post #373

Earlier quoted context omitted.

https://work.mercor.com/jobs/list_AAABljpKHPMmFMXrg2VM0qz4 https://benture.io/job/international-math-olympiad-participa... https://job-boards.greenhouse.io/xai/jobs/4538773007 And Outlier/Scale, which was bought by Meta (via Scale), had many IMO-required Math AI trainer jobs on LinkedIn. I can't find those historical ones though. I'm just one piece in the cog and this is an anecdote, but there was a huge upswing in I…

I would fully expect every IMO participant grinds IMO problems for months before the competition. I don't know why people hold training a model on like material as a negation of it's ability.

It shows models need RL for any new domain/level of expertise, which is contrary to what the marketers claim about LLMs and potential for AGI.

Re: OpenAI claims gold-medal performance at IMO 2025

#715

Post by famouswaffles »

YeGoblynQueenne wrote: post #370
modeless wrote: post #144

Earlier quoted context omitted.

Most evidence you have about the world is claims from other people, not direct experiment. There seems to be a thought-terminating cliche here on HN, dismissing any claim from employees of large tech companies. Unlike seemingly most here on HN, I judge people's trustworthiness individually and not solely by the organization they belong to. Noam Brown is a well known researcher in the field and I see no reason to doub…

A thought-terminating cliché? Not at all, certainly not when it comes to claims of technological or scientific breakthroughs. After all, that's partly why we have peer review and an emphasis on reproducibility. Until such a claim has been scrutinised by experts or reproduced by the community at large, it remains an unverified claim. >> Unlike seemingly most here on HN, I judge people's trustworthiness individually an…

They don't give a lot of details but they give enough for it to be pretty hard to say the claim is false but unfraudulent.

Some researchers got a breakthrough and decided to share right then rather than the months later it would take for a viable product. It happens, researchers are humans after all and i'm generally glad to take a peek at the actual frontier rather than what's behind by many months.

You can and it's fair to ignore such claims until that part but i think anything more than that is fairly uncharitable for the situation.

Re: OpenAI claims gold-medal performance at IMO 2025

#716

Post by fxj »

just tried this: take the graph of the functions x^n and exp(x) how many points of intersection do they have?

chatgpt gave me the wrong answer, it claimed 2 points of intersection, but for n=4 there are 3 as one can easily derive. one for negative x and 2 points for positive x because exp(x) is growing faster than x^4.

then i corrected it and said 3 points of intersection. it said yes and gaev me the 3 points. then i said no there are 4 points of intersection and it again explained to me that there are 2 points of intersection. which is wrong.

then i asked it how many points of intersection for n=e and it said: zero

well, exp(x)=x^e for x=e, isnt it?

Re: OpenAI claims gold-medal performance at IMO 2025

#717

Post by gus_massa »

tantalor wrote: post #667
gus_massa wrote: post #75

Earlier quoted context omitted.

In the IMO, the idea is that the first day you get P1, P2 and P3, and the second day you get P4, P5 and P6. Usually, ordered by difficulty, they are P1, P4, P2, P5, P3, P6. So, usually P1 is "easy" and P6 is very hard. At least that is the intended order, but sometime reality disagree. Edit: Fixed P4 -> P3. Thanks.

That's very silly. They should do the order like this: Day 1: P1 P3 P5 (odds) Day 2: P2 P4 P6 (evens) Then the problem # is the difficulty.

On one hand, it's very difficult to break traditions.

On the other hand, the order P1 P4 P2 P5 P3 P6 is not always true.

Usually there is only one problem of geometry per day.

Some problems involve a brilliant trick and another analyzing many cases. You don't want too "long" problems the same day. (Sometimes there is solution that the Jury didn't see and the problem changes it of made-up-category.)

Some problems are difficult but have a nice easy/medium intermediate step that assigns some points.

There are a lot of implicit restrictions that can affect the order of the problem.

Also, sometimes the Jury miscalculate how difficult is a problem and it's easier or more difficult than expected. Or the Jury completely miss an alternative easier solution.

The only sure part is the order that they are printed in the paper.

Re: OpenAI claims gold-medal performance at IMO 2025

#718

Post by peteyreplies »

xpressvideoz wrote: post #691
hislaziness wrote: post #658

Terence Tao on the matter - https://imgur.com/a/terence-tao-on-supposed-gold-imo-sMKP0bm

> there will be a proposal at some point to actually have an AI math Olympiad where at the same time as the human contestants get the actual Olympiad problems, AI’s will also be given the same problems, the same time period and the outputs will have to be graded by the same judges, which means that it’ll have be written in natural language rather than formal language.[1] Last month, Tao himself said that we can compa…

To be clear, AIs didn't beat IMO, or the best human competitors at IMO, on these terms.

Re: OpenAI claims gold-medal performance at IMO 2025

#719

Post by peteyreplies »

xpressvideoz wrote: post #116
tlb wrote: post #85

I encourage anyone who thinks these are easy high-school problems to try to solve some. They're published (including this year's) at https://www.imo-official.org/problems.aspx . They make my head spin.

I didn't know there were localized versions of the IMO problems. But now that I think of it, having versions of multiple languages is a must to remove the language barrier from the competitors. I guess having that many language versions (I see ~50 languages?) may make keeping the security of the problems considerably harder?

iirc, the IMO system automatically translates the questions into 50 languages, after they are entered in English.

Re: OpenAI claims gold-medal performance at IMO 2025

#720

Post by TestTime_9000 »

"The LLM system's core mechanism is probably a "propose-verify" loop that operates on a vocabulary of special tokens representing formal logic expressions. At inference time, it first proposes a new logical step by generating a sequence of these tokens into its context window, which serves as a computational workspace. It then performs a subsequent computational pass to verify if this new expression is a sound deduction from the preceding steps. This iterative cycle, learned from a vast corpus of synthetic proof traces, allows the model to construct a complete, validated formal argument. This process results in a system with abstract reasoning capabilities and functional soundness across domains that depend on reasoning, achieved at the cost of computation required for its extended inference time."
Post reply on HN