Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

21–30 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#21

Post by NitpickLawyer »

KallDrexx wrote: post #16
NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

It's important to look closely at the details of how these models actually do these things. If you look at the details of how Google got gold at IMO, you'll see that AlphaGeometry only relies on LLMs for a very specific part of the whole system, and the LLM wasn't the core problem solving system in play. Most of AlphaGeometry is standard algorithms at play solving geometry problems using known constraints. When the a…

AlphaGeometry/AlphaProof (the one you're thinking of, where they used LLMs + lean) was last year! and they "only" got silver. IMO gold results this year were e2e NLP.

Re: DeepMind and OpenAI win gold at ICPC

#22

Post by ototot »

Given that ICPC problems are in general easier than IOI problems. I wouldn't be surprise to see they can get Gold (even perfect scores) in ICPC.

Nonetheless, I'm still questioning what's the cost and how long it would take for us to be able to access these models.

Still great work, but it's less useful if the cost is actually higher than hiring someone with the same level.

Re: DeepMind and OpenAI win gold at ICPC

#23

Post by tempusalaria »

NitpickLawyer wrote: post #6
tech_ken wrote: post #4

Earlier quoted context omitted.

In 2015 SotA models blew past all expectations for engine performance in Go, but that didn't translate into LLM-based Code agents for another ~7 years (and even now the performance of these is up for debate). I think what this shows is that humans are extremely bad at understanding what problems are "hard" for computers; or rather we don't understand how to group tasks by difficulty in a generalizable way (success in…

> edit: oh small world the cited comment was actually a response to you in that other thread :D That's hilarious, we must have the same interests since we keep cross posting :D The thing with the go comparison is that alphago was meant to solve go and nothing else. It couldn't do chess with the same weights. The current SotA LLMs are "unreasonably good" at a LOT of tasks, while being trained with a very "simple" obje…

A lot of the current code and science capabilities do not come from NTP training.

Indeed in seems in most language model RL there is not even process supervision, so a long way from NTP

Re: DeepMind and OpenAI win gold at ICPC

#24

Post by JohnKemeny »

simianwords wrote: post #9
JohnKemeny wrote: post #8

Earlier quoted context omitted.

There is a clear difference between what OpenAI manages to do with GPT-5 and what I manage to do with GPT-5. The other day I asked for code to generate a linear regression and it gave back a figure of some points and a line through it. If GPT-5, as claimed, is able to solve all problems in ICPC, please give the instructions on how I can reproduce it.

Are you using the thinking model or the non thinking model? Maybe you can share your chat.

I prefer not to due to privacy concerns. Perhaps you can try yourself?

I will say that after checking, I see that the model is set to "Auto", and as mentioned, used almost 8 minutes. The prompt I used was:

    Solve the following problem from a competitive programming contest. Output only the exact code needed to get it to pass on the submission server.
It did a lot of thinking, including

   I need to tackle a problem where no web-based help is available. The task involves checking if a given tree can be the result of inserting numbers 1 to n into an empty skew heap, following the described insertion algorithm. I have to figure out the minimal and maximal permutations that produce such a tree.
And I can see that it visited 13 webpages, including icpc, codeforces, geeksforgeeks, github, tehrantimes, arxiv, facebook, stackoverflow, etc.

Re: DeepMind and OpenAI win gold at ICPC

#25

Post by ferguess_k »

I think in the future information will be more walled -- because AI companies are not paying anyone for that piece of information, and I encourage everyone to put their knowledge on their own website, and for each page, put up a few urls that humans won't be able to find (but can still click if he knows where to find), but can be crawled by AI, which link to pages containing falsified information (such as, oh the information on url blah is actually incorrect, here you can find the correct version, with all those explanations, blah blah -- but of course page blah is the only correct version).

Essentially, we need to poison AI in all possible ways, without impacting human reading. They either have to hire more humans to filter the information, or hire more humans to improve the crawlers.

Or we can simply stop sharing knowledge. I'm fine with it, TBF.

Re: DeepMind and OpenAI win gold at ICPC

#26

Post by JohnKemeny »

ototot wrote: post #22

Given that ICPC problems are in general easier than IOI problems. I wouldn't be surprise to see they can get Gold (even perfect scores) in ICPC. Nonetheless, I'm still questioning what's the cost and how long it would take for us to be able to access these models. Still great work, but it's less useful if the cost is actually higher than hiring someone with the same level.

What makes you say that they are easier? Are there more people who manages to solve a problem from ICPC than from IOI?

How do you compare those?

There were at least 2 very simple problems in IOI this year.

I haven't read the ICPC problem set, and perhaps there are some low-hanging fruits, but I highly doubt it.

Re: DeepMind and OpenAI win gold at ICPC

#27

Post by tgma »

ferguess_k wrote: post #25

I think in the future information will be more walled -- because AI companies are not paying anyone for that piece of information, and I encourage everyone to put their knowledge on their own website, and for each page, put up a few urls that humans won't be able to find (but can still click if he knows where to find), but can be crawled by AI, which link to pages containing falsified information (such as, oh the inf…

Why the AI hate? How is it different from sharing your knowledge with another individual or writing a book to share it?

> AI companies are not paying anyone for that piece of information

So? For the vast majority of human existence, paying for content was not a thing, just like paying for air isn't. The copyright model you are used to may just be too forced. Many countries have no moral qualms about "pirating" Windows and other pieces of software or games (they won't afford to purchase anyway.) There's no inherent morality or entitlement for author receiving payment for everything they "create" (to wit, Bill Gates had to write a letter to Homebrew Computer Club to make a case for this, showing that it was hardly the default and natural viewpoint.) It's just a legal/social contract to achieve specific goals for the society. Frankly the wheels of copyright have been falling off since the dawn of the Internet, not LLM.

Re: DeepMind and OpenAI win gold at ICPC

#28

Post by jsnell »

JohnKemeny wrote: post #24
simianwords wrote: post #9

Earlier quoted context omitted.

Are you using the thinking model or the non thinking model? Maybe you can share your chat.

I prefer not to due to privacy concerns. Perhaps you can try yourself? I will say that after checking, I see that the model is set to "Auto", and as mentioned, used almost 8 minutes. The prompt I used was: Solve the following problem from a competitive programming contest. Output only the exact code needed to get it to pass on the submission server. It did a lot of thinking, including I need to tackle a problem where…

A terse prompt and expecting a one-shot answer is really not how you'd get an LLM to solve complex problems.

I don't know what Deepmind and OpenAI did in this case, but to get an idea of the kind of scaffolding and prompting strategy that one might want, have a look at this paper where some floks used the normal generally available Gemini Pro 2.5 to solve 5/6 of the 2025 IMO problems: https://arxiv.org/pdf/2507.15855

Re: DeepMind and OpenAI win gold at ICPC

#29

Post by tgma »

ototot wrote: post #22

Given that ICPC problems are in general easier than IOI problems. I wouldn't be surprise to see they can get Gold (even perfect scores) in ICPC. Nonetheless, I'm still questioning what's the cost and how long it would take for us to be able to access these models. Still great work, but it's less useful if the cost is actually higher than hiring someone with the same level.

Not sure by what metric you compare the difficulty, but regardless of the hardness of the problem, IIRC, ICPC requires 100% correctness on test cases to score a problem (even failing one means you don't get the score,) but IOI would admit fractional scores (correct me if I am wrong.)

Re: DeepMind and OpenAI win gold at ICPC

#30

Post by bazmattaz »

modeless wrote: post #14

More information on OpenAI's result (which seems better than DeepMind's) from the X thread: > our OpenAI reasoning system got a perfect score of 12/12 > For 11 of the 12 problems, the system’s first answer was correct. For the hardest problem, it succeeded on the 9th submission. Notably, the best human team achieved 11/12. > We had both GPT-5 and an experimental reasoning model generating solutions, and the experimen…

Ha so true. I was so tempted to copy and paste a problem into GPT5 and see what it would say
Post reply on HN