Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

171–180 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#171

Post by sidibe »

sameermanek wrote: post #86

Whats the point? These models are still unreliable in every day work. And they're getting fat! For a moment, they were getting cheaper, but now they are only getting bigger and this is not going to be cheap in the future. The point is, what are we investing a trillion dollars in?

Unreliable doesn't mean unusable. I'm finding it harder and harder to believe people are actually trying to use them and saying they are useless.

If you can chop your problem up and give little tedious parts of your bigger task it's starting like doing code review for a new grad instead of coding. And they're getting more reliable and the parts you give it can be bigger and bigger. I wish there was a way to stop this but I don't think it's going to.

Re: DeepMind and OpenAI win gold at ICPC

#172

Post by notemap »

patrickhogan1 wrote: post #157
patrickhogan1 wrote: post #154

Earlier quoted context omitted.

1. What was your prompt? 2. Why did you give it to GPT-5 instead of GPT-5 Thinking or GPT-5 Pro?

Here is the prompt I just gave to GPT-5 Pro - its chugging on it. Not sure if it will succeed. Let's see what happens. I did think about converting the PDF to markdown, but figured this prompt is more fair. - You are a gold level math olympiad competitor participating in the ICPC 2025 Baku competition. You will be given a competitive programming problem to solve completely. All problems are located at the following U…

If we're benchmarking problems, mind trying out this problem on Pro if you're willing to spare the compute?

https://www.acmicpc.net/problem/33797

I have the 20$ plan and I think I found a weird bug, at least with the thinking version. It gets stuck in the same local minima super quickly, even though the "fake solution" is easily disproved on random tests.

It's at the point where sometimes I've fed it the editorial and it still converges to the fake solution.

https://chatgpt.com/share/68c8b2ef-c68c-8004-8006-595501929f...

I'm sure that the model is capable of solving it, but seriously I've tried across multiple generations (since about when o3 came out) to get GPT to solve this problem and it's not hampered by its innate ability I don't think, it literally just refuses to think critically about the problem. Maybe with better prompting it doesn't get stuck as hard?

Re: DeepMind and OpenAI win gold at ICPC

#173

Post by tdb7893 »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

As someone who has been to the ICPC finals around a decade ago I agree that the limited time is really the big problem that these machine learning models don't really experience in the same way. Though that being said these problems are hard, the actual coding of the algorithms is pretty easy (most of the questions use one of a handful of algorithms that you've implemented a hundred times by the time you're in the finals) but recognizing which one will actually solve the problem correctly is not obvious at all. I know a lot of people that struggled in their undergrad algorithms class and I think a lot of those people given the ICPC finals problems would struggle even with being able to research.

Re: DeepMind and OpenAI win gold at ICPC

#174

Post by Davidzheng »

jaggs wrote: post #13

I think it's becoming clear that these mega AI corps are juggling with their models at inference time to produce unrealistically good results. By that it seems that they're just cranking up the compute beyond reasonable levels in order to gain PR points against each other. The fact is most ordinary mortals never get access to a fraction of that kind of power, which explains the commonly reported issues with AI models…

I think part of it depends on whether you see AI progress as research or product.

Re: DeepMind and OpenAI win gold at ICPC

#175

Post by dddgghhbbfblk »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

I think that's because the framing around this (and similar stories about eg IMO performances) is imo slightly wrong. It's not interesting that they can get a gold medal in the sense of trying to rank them against human competitors. As you say, the direct comparisons are, while not entirely meaningless, at least very hard to interpret in the best of cases. It's very much an apples to oranges situation.

Rather, the impressive thing is simply that an AI is capable of solving these problems at all. These are novel (ie not in training set) problems that are really hard and beyond the ability of most professional programmers. The "gold medal" part is informative more in the sense that it gives an indication of how many problems the AI was able to solve & how well it was able to do them.

When talking with some friends about chatgpt just a couple years ago I remember being very confident that there was no way this technology would be able to solve this kind of novel, very challenging reasoning problem, and that there was no way it would be able to solve IMO problems. It's remarkable how quickly I've been proven wrong.

Re: DeepMind and OpenAI win gold at ICPC

#176

Post by m3kw9 »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

the end game is that running similar tasks at any moment time and place.

Re: DeepMind and OpenAI win gold at ICPC

#177

Post by m3kw9 »

NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

the wall is how we need to throw trillions of hardware to do "breakthroughs", LLM uses the same algorthm from last few years. We need a new algorthm breakthrough otherwise buying hardware to increase intelligence isn't scalable.

Re: DeepMind and OpenAI win gold at ICPC

#180

Post by dwohnitmok »

Swizec wrote: post #155
throw310822 wrote: post #133

Earlier quoted context omitted.

I think you missed that the whole point of this race was: "did we build a vehicle faster than a horse, yes/no?" Which matters a lot when horses are the fastest land vehicle available. (We're so used to thinking of horses as a quaint and slow mean of transport that maybe we don't realize that for millennia they've been the fastest possible way to get from one place to another.)

> "did we build a vehicle faster than a horse, yes/no?" Yeah fair. There's also that famous human vs horse race that happens every few years. So far humans keep winning (because it's long distance)

If you're talking about the Man versus Horse Marathon (https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon) it's the other way around. Overwhelmingly the horses win. Only occasionally does the human.
Post reply on HN