DeepMind and OpenAI win gold at ICPC • Hacker News Board

Re: DeepMind and OpenAI win gold at ICPC

#111

Post by huflungdung » Wed, Sep 17, 2025, 10:24 PM UTC

asboans wrote: post #104

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

[dead]

Re: DeepMind and OpenAI win gold at ICPC

#112

Post by lbrandy » Wed, Sep 17, 2025, 10:30 PM UTC

asboans wrote: post #104

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

I was struck how the argument is also isomorphic to how we talked about computers and chess. We're at the stage where we are arguing the computer isn't _really_ understanding chess, though. It's just doing huge amounts of dumb computation with huge amounts of opening book and end tables and no real understanding, strategy or sense of whats going on.

Even though all the criticism were, in a sense, valid, in the end none of it amounted to a serious challenge to getting good at the task at hand.

Re: DeepMind and OpenAI win gold at ICPC

#113

Post by LaffertyDev » Wed, Sep 17, 2025, 10:31 PM UTC

asboans wrote: post #104

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

I don’t think you’ll find many race tracks that permit horses and cars to compete together.

(I did enjoy the sarcasm, though!)

Re: DeepMind and OpenAI win gold at ICPC

#114

Post by Ylpertnodi » Wed, Sep 17, 2025, 10:32 PM UTC

amluto wrote: post #87

HarHarVeryFunny wrote: post #57

ICPC = The International Collegiate Programming Contest. These are college level programmers, not elite competitive programmers. Apparently Gemini solved one problem (running on who knows what kind of cluster) by burning 30 min of "thinking" time on it, and at a cost that Google have declined to provide. According to one prior competition paricipant, writing in the comments section of this ArsClasica coverage, each y…

These are college-student or occasionally grad-school programmers who qualified to enter the ICPC World Finals , generally by performing sufficiently well at a regional championship to qualify. You can read actual rules here (see "Advancing to the ICPC World Finals"): https://icpc.global/regionals/rules I don't know what you mean by "elite", and there are certainly plenty of teams at the World Finals that are not esp…

> There are many people and teams at the Olympics who are not remotely competitive with the winners.

And yet, they are so much closer to the winners than the people that came 11th, 12th etc.

Re: DeepMind and OpenAI win gold at ICPC

#115

Post by Swizec » Wed, Sep 17, 2025, 10:36 PM UTC

asboans wrote: post #104

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

> Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses

Yes. That’s why cars don’t compete in equestrian events and horses don’t go to F1 races.

This non-controversial surely? You want different events for humans, humans + computers, and just computers.

Notice that self driving cars have separate race events from both horses and human-driven cars.

Re: DeepMind and OpenAI win gold at ICPC

#116

Post by z7 » Wed, Sep 17, 2025, 10:42 PM UTC

Current cope collection:

- It's not a fair match, these models have more compute and memory than humans

- Contestants weren't really elite, they're just college level programmers, not the world's best

- This doesn't matter for the real world, competitive programming is very different from regular software engineering

- It's marketing, they're just cranking up the compute to unrealistic levels to gain PR points

- It's brute force, not intelligence

Re: DeepMind and OpenAI win gold at ICPC

#117

Post by bgwalter » Wed, Sep 17, 2025, 10:42 PM UTC

asboans wrote: post #104

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

The massive amounts of compute power is not the major issue. The major issue is unlimited amount of reference material.

If a human can look up similar previous problems just as the "AI" can, it is a huge advantage.

Syzygy tables in chess engines are a similar issue. They allow perfect play, and there is no reason why a computer gets them and a human does not (if you compare humans against chess engines). Humans have always worked with reference material for serious work.

Re: DeepMind and OpenAI win gold at ICPC

#118

Post by 77pt77 » Wed, Sep 17, 2025, 10:46 PM UTC

paxys wrote: post #62

NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

My response simply is that performance in coding competitions such as ICPC is a very different skillset than what is required in a regular software engineering job. GPT-5 still cannot make sense of my company's legacy codebase even if asked to do the most basic tasks that a new grad out of college can figure out in a day or two. I recently asked it to fix a broken test (I had messed with it by changing one single ass…

> it declared "success" by deleting the entire test suite.

The paperclip trivial solution!

Re: DeepMind and OpenAI win gold at ICPC

#119

Post by warkdarrior » Wed, Sep 17, 2025, 10:47 PM UTC

zeroonetwothree wrote: post #72

modeless wrote: post #67

Earlier quoted context omitted.

It doesn't matter how many instances were running. All that matters is the wall clock time and the cost. The fact that they don't disclose the cost is a clue that it's probably outrageous today. But costs are coming down fast. And hiring a team of these guys isn't exactly cheap either.

Human teams are limited to three people. So why doesn’t it matter how many instances they used?

I bet with human teams it'll take longer to solve a problem the more people you have on the team.

Re: DeepMind and OpenAI win gold at ICPC

#120

Post by somewholeother » Wed, Sep 17, 2025, 10:47 PM UTC

dist-epoch wrote: post #81

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

If you want to play that game, let's compute how much energy was spent to grow, house and educate one team since they were born, over 20 years against how much was spent training the model.

This is a fair analogy, but let's also consider that these human beings weren't designed with the express purpose of becoming experts in their field and performing in this way for this specific purpose (albeit in a generalist manner).

We are most definitely in agreement about the folly of comparing the abilities of LLMs to humans, since LLMs are to a greater extent the product of much collective human endeavour. "Living memories" would perhaps be a better description of their current state, and their resultant impact on the human psyche.