OpenAI claims gold-medal performance at IMO 2025 • Hacker News Board

Re: OpenAI claims gold-medal performance at IMO 2025

#611

Post by 13years » Sat, Jul 19, 2025, 11:07 PM UTC

highwaylights wrote: post #438

transcriptase wrote: post #383

Earlier quoted context omitted.

Having spent tens of thousands of hours contributing to scientific discovery by reading dense papers for a single piece of information, reverse engineering code written by biologists, and tweaking graphics to meet journal requirements… I can say with certainty it’s already contributing by allowing scientists to spend time on science versus spending an afternoon figuring out which undocumented argument in a R package…

This. Even if LLM’s ultimately hit some hard ceiling as substantially-better-Googling-automatons they would already accelerate all thought-based work across the board, and that’s the level they’re already at now (arguably they’re beyond that). We’re already at the point where these tools are removing repetitive/predictable tasks from researchers (and everyone else), so clearly they’re already accelerating research.

Not sure how you get around the contamination problems. I use these everyday and they are extremely problematic about making errors that are hard to perceive.

They are not reliable tools for any tasks that require accurate data.

Re: OpenAI claims gold-medal performance at IMO 2025

#612

Post by kappi » Sat, Jul 19, 2025, 11:07 PM UTC

yongjik wrote: post #559

kappi wrote: post #472

Earlier quoted context omitted.

[flagged]

All the past IMO problems are known to public and contestants practice on them. If solving an IMO problem is the simple matter of "looking at all the past problems and apply the same pattern," you'd expect human contestants to do a lot better.

I think you haven't gone thru AMC8, AMC10, AIME competitions. If you are so confident, try giving an unsolved math problem outside the high school math competitions.

Re: OpenAI claims gold-medal performance at IMO 2025

#613

Post by Workaccount2 » Sat, Jul 19, 2025, 11:08 PM UTC

Invictus0 wrote: post #58

npinsker wrote: post #31

Earlier quoted context omitted.

The trouble is, getting an IMO gold medal is much easier (by frequency) than being the #1 Go player in the world, which was achieved by AI 10 years ago. I'm not sure it's enough to just gesture at the task; drilling down into precisely how it was achieved feels important. (Not to take away from the result, which I'm really impressed by!)

The "AI" that won Go was Monte Carlo tree search on a neural net "memory" of the outcome of millions of previous games; this is a LLM solving open ended problems. The tasks are hardly even comparable.

Nothing that uses a mathematical model for solving a problem will ever reason because reasoning can only be done by things we don't understand...

Re: OpenAI claims gold-medal performance at IMO 2025

#614

Post by torginus » Sat, Jul 19, 2025, 11:10 PM UTC

krackers wrote: post #589

torginus wrote: post #580

Earlier quoted context omitted.

In transformers generating each token takes the same amount of time, regardless of how much meaning it carries. By cutting out the filler from the text, you get a huge speedup.

Except generating more tokens also effectively extends the computational power beyond the depth of the circuit, which is why chain of thought works in the first place. Even sampling only dummy tokens that don't convey anything still provides more computational power.

I mean, generating more tokens means you use more computing power, and there's som e evidence that not all of these filler words go to waste (esp since they are not really words, but vectors that can carry latent meaning), as models tend to become smarter when allowed to generate a lot of heeming and hawing.

It's been proven that this accidental computation is actually helping CoT models, but they're not supposed to work like that - they're supposed to generate logical observations and use said observations to work further towards the goal (and they primarily do do that).

Considering filler tokens occupy context space and are less useful than meaningful tokens, a model that tries to maximize useful results per amount of compute, you'd want a terse context window without any fluff.

Re: OpenAI claims gold-medal performance at IMO 2025

#615

Post by atleastoptimal » Sat, Jul 19, 2025, 11:13 PM UTC

bgwalter wrote: post #606

atleastoptimal wrote: post #412

>AI model performs astounding feat everyone claimed was impossible or won’t be achieved for a while >Commenters on HN claim it must not be that hard, or OpenAI is lying, or cheated. Anything but admit that it is impressive Every time on this site lol. A lot of people here have an emotional aversion to accepting AI progress. They’re deep in the bargaining/anger/denial phase.

Thank you for this pop psychology evaluation. It could have been written by an an "AI".

Simply stating what is going on in the comments, as happens any time AI hits a previously thought to be impossible or far-off milestone.

Re: OpenAI claims gold-medal performance at IMO 2025

#616

Post by joe_the_user » Sat, Jul 19, 2025, 11:16 PM UTC

YeGoblynQueenne wrote: post #115

dylanbyte wrote: post #16

These are high school level only in the sense of assumed background knowledge, they are extremely difficult. Professional mathematicians would not get this level of performance, unless they have a background in IMO themselves. This doesn’t mean that the model is better than them in math, just that mathematicians specialize in extending the frontier of math. The answers are not in the training data. This is not a mode…

>> This is not a model specialized to IMO problems. How do you know?

Yeah, looking at the GP ... say a sequence of things that are true and plausible. That add your strong, unsupported claim at the end. I remember the approach from when I studied persuasion techniques...

Re: OpenAI claims gold-medal performance at IMO 2025

#617

Post by atleastoptimal » Sat, Jul 19, 2025, 11:18 PM UTC

chairhairair wrote: post #77

OpenAI simply can’t be trusted on any benchmarks: https://news.ycombinator.com/item?id=42761648

On OpenAI's own released papers they show Anthropic's models performing better than their own. They tend to be pretty transparent and reliable in honesty in their benchmarks.

The thing is, only leading AI companies and big tech have the money to fund these big benchmarks and run inference on them. As long as the benchmarks are somewhat publicly available and vetted by reputable scientists/mathematicians it seems reasonable to believe they're trustworthy.

Re: OpenAI claims gold-medal performance at IMO 2025

#618

Post by raincole » Sat, Jul 19, 2025, 11:19 PM UTC

amelius wrote: post #596

If someone told me this say, 10 or 20 years ago, I would have assumed this was worthy of a Nobel/Turing prize ...

Early machine learning researchers literally got Nobel Prize last year. Clearly not every incremental step of progress merits a Nobel.

Re: OpenAI claims gold-medal performance at IMO 2025

#619

Post by sigbottle » Sat, Jul 19, 2025, 11:20 PM UTC

kevinventullo wrote: post #551

sealeck wrote: post #461

Earlier quoted context omitted.

The Putnam and IMO are quite different. I would suggest the IMO is probably harder...

I would disagree; the IMO depends only on late middle school/early high school level mathematics (geometry, gcd, functions) while Putnam typically depends on late high school/early college-level mathematics (integrals, limits, matrices).

Well I come from the competitive programming sphere, and I would say that IOI is harder than ICPC.

When you don't know that many things, that's when creativity shines, and there are some truly genuinely shocking IOI problems.

ICPC (well, in recent years they've gotten slightly better) is pretty well known as a knowledge-heavy implementation contest. Many teams get the experience that they mind-solved a lot more problems but couldn't implement them in time. Typing up the maxflow template for the 25th time for a series of collegiate-level but ultimately standard reductions isn't that inspiring.

My favorite problems are those you can derive from basic techniques but come up with scaffolding that is truly elegant. I've set some of them myself, which have stumped some famous people you may know :)

---

I guess my point is that I can see people feeling about Putnam the same way.

Re: OpenAI claims gold-medal performance at IMO 2025

#620

Post by fredoliveira » Sat, Jul 19, 2025, 11:23 PM UTC

dataflow wrote: post #469

NooneAtAll3 wrote: post #449

Earlier quoted context omitted.

I thought single digit means single significant digit, aka rounding to 10%?

Wasn't 16% the example they were talking about? Isn't that two significant digits? And 16% very much feels ridiculous to a reader when they could've just said 15%.

My interpretation was that Yudkowski simply doubled Christiano's guess of 8% (as one might say in conversation "oh it's at least double that", but using the actual number)