Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

61–70 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#61

Post by blibble »

tgma wrote: post #47
program_whiz wrote: post #46

Earlier quoted context omitted.

Its different because the AI model will then automate the use of that knowledge, which for most people in this forum is how they make their livelihood. If OpenAI were making robots to replace plumbers, I wouldn't be surprised when plumbers said "we should really stop giving free advice and training to these robots." Its in the worker's best interest to avoid getting undercut by an automated system that can only be bu…

> plumbers I have received free advice that reduced future need from such actual plumbers (and mechanics and others for that matter) > we should really stop giving free advice and training to these robots People routinely freely give advice and teach students, friends, potential competitors, actual competitors, etc on this same forum. Robots? Many also advocate for immigration and outsourcing, presumably because they…

the AI isn't malevolent (... yet)

but the companies operating it certainly are

they have no concept of consent

they take anything and everything, regardless of copyright or license, with no compensation to the authors

and then use it to directly compete with those they ripped off

not to mention shoving their poor quality generated slop everywhere they can possibly manage, regardless of ethics, consent or potential consequences

children should not be supplied a sycophantic source of partial truths that has been instructed to pretend to be their friend

this is text book malevolence

Re: DeepMind and OpenAI win gold at ICPC

#62

Post by paxys »

NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

My response simply is that performance in coding competitions such as ICPC is a very different skillset than what is required in a regular software engineering job. GPT-5 still cannot make sense of my company's legacy codebase even if asked to do the most basic tasks that a new grad out of college can figure out in a day or two. I recently asked it to fix a broken test (I had messed with it by changing one single assertion) and it declared "success" by deleting the entire test suite.

Re: DeepMind and OpenAI win gold at ICPC

#63

Post by paxys »

JohnKemeny wrote: post #8
NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

There is a clear difference between what OpenAI manages to do with GPT-5 and what I manage to do with GPT-5. The other day I asked for code to generate a linear regression and it gave back a figure of some points and a line through it. If GPT-5, as claimed, is able to solve all problems in ICPC, please give the instructions on how I can reproduce it.

Yeah, until OpenAI says "we pasted the questions from ICPC into chatgpt.com and it scored 12/12" the average user isn't really going to be able to reproduce their results.

Re: DeepMind and OpenAI win gold at ICPC

#64

Post by paladin314159 »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

> I think that, if you gave me the ability to search the pre-contest Internet and a week to prepare my submissions, I would be kind of embarrassed if I didn't get gold, and I'd find the contest to be rather less interesting than I would find the real thing.

I don't know what your personal experience with competitive programming is, so your statement may be true for yourself, but I can confidently state that this is not true for the VAST majority of programmers and software engineers.

Much like trying to do IMO problems without tons of training/practice, the mid-to-hard problems in the ICPC are completely unapproachable to the average computer science student (who already has a better chance than the average software engineer) in the course of a week.

In the same way that LLMs have memorized tons of stuff, the top competitors capable of achieving a gold medal at the ICPC know algorithms, data structures, and how to pattern match them to problems to an extreme degree.

Re: DeepMind and OpenAI win gold at ICPC

#65

Post by tgma »

blibble wrote: post #61
tgma wrote: post #47

Earlier quoted context omitted.

> plumbers I have received free advice that reduced future need from such actual plumbers (and mechanics and others for that matter) > we should really stop giving free advice and training to these robots People routinely freely give advice and teach students, friends, potential competitors, actual competitors, etc on this same forum. Robots? Many also advocate for immigration and outsourcing, presumably because they…

the AI isn't malevolent (... yet) but the companies operating it certainly are they have no concept of consent they take anything and everything, regardless of copyright or license, with no compensation to the authors and then use it to directly compete with those they ripped off not to mention shoving their poor quality generated slop everywhere they can possibly manage, regardless of ethics, consent or potential co…

> but the companies operating it certainly are

Which ones in particular? Is your belief all that are companies are inherently malevolent? If not why don't you start one that is not? What's stopping you?

Re: DeepMind and OpenAI win gold at ICPC

#66

Post by mannycalavera42 »

HarHarVeryFunny wrote: post #57

ICPC = The International Collegiate Programming Contest. These are college level programmers, not elite competitive programmers. Apparently Gemini solved one problem (running on who knows what kind of cluster) by burning 30 min of "thinking" time on it, and at a cost that Google have declined to provide. According to one prior competition paricipant, writing in the comments section of this ArsClasica coverage, each y…

I've competed in these contest before. There are probably more difficult than what we can call _elite_ competitive programmer

note: my team only passed the first 2 rounds, far from bragging about my skills here :)

Re: DeepMind and OpenAI win gold at ICPC

#67

Post by modeless »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

It doesn't matter how many instances were running. All that matters is the wall clock time and the cost.

The fact that they don't disclose the cost is a clue that it's probably outrageous today. But costs are coming down fast. And hiring a team of these guys isn't exactly cheap either.

Re: DeepMind and OpenAI win gold at ICPC

#68

Post by zeroonetwothree »

Davidzheng wrote: post #60
jaggs wrote: post #13

I think it's becoming clear that these mega AI corps are juggling with their models at inference time to produce unrealistically good results. By that it seems that they're just cranking up the compute beyond reasonable levels in order to gain PR points against each other. The fact is most ordinary mortals never get access to a fraction of that kind of power, which explains the commonly reported issues with AI models…

Ok but if they can pump those compute and get science/math advancements it's worth something even if the costs are very high

ICPC problems are about as far from scientific advancements as a spelling bee is from Shakespeare.

(I’m a former ICPC competitor)

Re: DeepMind and OpenAI win gold at ICPC

#69

Post by zeroonetwothree »

andy12_ wrote: post #37
jaggs wrote: post #33

Earlier quoted context omitted.

Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?

When I say 10 times cheaper, I mean when comparing models of the same capabilities. The kind of performance you get now for a 200$ subscription, a year ago probably would have costed 2000$.

I don’t believe that current models are 1000x better than the initial ChatGPT release. What metric are you using?

Re: DeepMind and OpenAI win gold at ICPC

#70

Post by stevesimmons »

ototot wrote: post #51
tgma wrote: post #49

Earlier quoted context omitted.

> Because I'm a ICPC medalist (not this year though) but not a IOI medalist. Isn't getting a medal a function of your ranking, not score, in both cases? If so, that does not prove much about the difficulty of either.

OK. I think my opinion and definition on "easier" is indeed vague. For "easier", I'm only comparing the thinking difficulty. Yes, medal is function of ranking but not difficulty. Nonetheless, I would say that IOI more focus on thinking, which I to some degree is not that good at, while ICPC is more like a mix thinking and implementing. Therefore, my ability to implement stuff can improve my ICPC ranking but not IOI.

As a former ICPC winner, I'd say ICPC is mainly a test of teamwork, given the format of the competition (3 team mates, one computer, scoring that rewards clean solutions submitted quicker, tackled in the optimal order for your three sets of skills, etc).

Sure, you need to be individually good at thinking, etc. But the difference between 1st and places further down the ranking is teamwork.

Post reply on HN