DeepMind and OpenAI win gold at ICPC • Hacker News Board

Re: DeepMind and OpenAI win gold at ICPC

#81

Post by dist-epoch » Wed, Sep 17, 2025, 9:21 PM UTC

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

If you want to play that game, let's compute how much energy was spent to grow, house and educate one team since they were born, over 20 years against how much was spent training the model.

Re: DeepMind and OpenAI win gold at ICPC

#82

Post by jamespo » Wed, Sep 17, 2025, 9:21 PM UTC

tgma wrote: post #39

bgwalter wrote: post #34

Earlier quoted context omitted.

Companies valued at $300 billion or more are not another individual and people are not "sharing" their works. The companies are stealing them. For the majority of interesting output people have paid for art, music, software, journalism. But you know that already and are justifying the industry that pays your bills.

> valued at $300 billion Irrelevant really. Invoking this in the argument shows the basis is jealousy. They are clearly valued as such not because they collected all the data and stored in some database. Your local library is not worth 300 billion. > For the majority of interesting output people have paid for art, music, software, journalism Absolutely and demonstrably false. Music and art predate Copyright by hundre…

Copyright is not the same as paying for it

Re: DeepMind and OpenAI win gold at ICPC

#83

Post by dist-epoch » Wed, Sep 17, 2025, 9:25 PM UTC

HarHarVeryFunny wrote: post #57

ICPC = The International Collegiate Programming Contest. These are college level programmers, not elite competitive programmers. Apparently Gemini solved one problem (running on who knows what kind of cluster) by burning 30 min of "thinking" time on it, and at a cost that Google have declined to provide. According to one prior competition paricipant, writing in the comments section of this ArsClasica coverage, each y…

Let's bookmark this comment and check again next year, if the freely available models will be able to do it for a few dollars.

Re: DeepMind and OpenAI win gold at ICPC

#84

Post by ben_w » Wed, Sep 17, 2025, 9:26 PM UTC

zeroonetwothree wrote: post #72

modeless wrote: post #67

Earlier quoted context omitted.

It doesn't matter how many instances were running. All that matters is the wall clock time and the cost. The fact that they don't disclose the cost is a clue that it's probably outrageous today. But costs are coming down fast. And hiring a team of these guys isn't exactly cheap either.

Human teams are limited to three people. So why doesn’t it matter how many instances they used?

All instances of any given model are kinda the same, for lack of a better word, "person": same knowledge, same skills, same failings.

Re: DeepMind and OpenAI win gold at ICPC

#85

Post by mrkeen » Wed, Sep 17, 2025, 9:28 PM UTC

paxys wrote: post #62

NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

My response simply is that performance in coding competitions such as ICPC is a very different skillset than what is required in a regular software engineering job. GPT-5 still cannot make sense of my company's legacy codebase even if asked to do the most basic tasks that a new grad out of college can figure out in a day or two. I recently asked it to fix a broken test (I had messed with it by changing one single ass…

Similar experience with windsurf.

I had a class of 5 or so test methods - ABCDE. I asked it to fix C, so it started typing out B token-by-token underneath C, such that my source file was now ABCBDE.

I don't think I'm smart enough to get it to do coding activities.

Re: DeepMind and OpenAI win gold at ICPC

#86

Post by sameermanek » Wed, Sep 17, 2025, 9:32 PM UTC

Whats the point? These models are still unreliable in every day work. And they're getting fat! For a moment, they were getting cheaper, but now they are only getting bigger and this is not going to be cheap in the future. The point is, what are we investing a trillion dollars in?

Re: DeepMind and OpenAI win gold at ICPC

#87

Post by amluto » Wed, Sep 17, 2025, 9:34 PM UTC

HarHarVeryFunny wrote: post #57

ICPC = The International Collegiate Programming Contest. These are college level programmers, not elite competitive programmers. Apparently Gemini solved one problem (running on who knows what kind of cluster) by burning 30 min of "thinking" time on it, and at a cost that Google have declined to provide. According to one prior competition paricipant, writing in the comments section of this ArsClasica coverage, each y…

These are college-student or occasionally grad-school programmers who qualified to enter the ICPC World Finals, generally by performing sufficiently well at a regional championship to qualify. You can read actual rules here (see "Advancing to the ICPC World Finals"):

https://icpc.global/regionals/rules

I don't know what you mean by "elite", and there are certainly plenty of teams at the World Finals that are not especially competitive, and there certainly many elite programers who don't qualify for various reasons (most obviously by being the wrong age or not in the right stage of school or having already attended too many times), but I find it hard to believe that there aren't enough "elite" programmers present to make the winning teams be genuinely elite.

Compare to, say, the Olympics or pretty much any academic olympiad. There are many people and teams at the Olympics who are not remotely competitive with the winners.

Re: DeepMind and OpenAI win gold at ICPC

#88

Post by avcxz » Wed, Sep 17, 2025, 9:34 PM UTC

amluto wrote: post #80

paladin314159 wrote: post #64

Earlier quoted context omitted.

> I think that, if you gave me the ability to search the pre-contest Internet and a week to prepare my submissions, I would be kind of embarrassed if I didn't get gold, and I'd find the contest to be rather less interesting than I would find the real thing. I don't know what your personal experience with competitive programming is, so your statement may be true for yourself, but I can confidently state that this is n…

> I can confidently state that this is not true for the VAST majority of programmers and software engineers. That may well be true. I think it's even more true in cases where the user is not a programmer by profession. I once watched someone present their graduate-level research in a different field and explain how they had solved a real-world problem in their field by writing a complicated computer program full of c…

> That's a whole field of wonderful computer science / math that is mostly usable via a couple of extraordinarily expensive closed-source offerings.

Take a look at Google OR-Tools: https://developers.google.com/optimization/

Re: DeepMind and OpenAI win gold at ICPC

#89

Post by anthonypasq » Wed, Sep 17, 2025, 9:34 PM UTC

paxys wrote: post #63

JohnKemeny wrote: post #8

Earlier quoted context omitted.

There is a clear difference between what OpenAI manages to do with GPT-5 and what I manage to do with GPT-5. The other day I asked for code to generate a linear regression and it gave back a figure of some points and a line through it. If GPT-5, as claimed, is able to solve all problems in ICPC, please give the instructions on how I can reproduce it.

Yeah, until OpenAI says "we pasted the questions from ICPC into chatgpt.com and it scored 12/12" the average user isn't really going to be able to reproduce their results.

the average person doesnt need to do that. The benchmark for "is this response accurate and personable enough" on any basic chat app has been saturated for at least a year at this point.

Re: DeepMind and OpenAI win gold at ICPC

#90

Post by Imnimo » Wed, Sep 17, 2025, 9:37 PM UTC

My understanding is that the way they do this is have some number of model instances generating solution proposals, and then another model which chooses which candidates to submit.

I haven't been able to find information on how many proposals were generated before a solution was chosen to submit. I'm curious to know whether this is "you can get ICPC gold medal performance with a handful of GPT-5 instances" or "you will drown yourself in API credit debt if you try this".

Still extremely impressive either way.