Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

181–190 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#181

Post by thrwaway55 »

in-silico wrote: post #156
Swizec wrote: post #115

Earlier quoted context omitted.

> Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses Yes. That’s why cars don’t compete in equestrian events and horses don’t go to F1 races. This non-controversial surely? You want different events for humans, humans + computers, and just computers. Notice that self driving cars have separate race events from both horses and human-driven cars…

The point is that up until now, humans were the best at these competitions, just like horses were the best at racing up until cars came around. The other commenter is pointing out how ridiculous it would be for someone to downplay the performance of cars because they did it differently from horses. It doesn't matter if they did it using different methods, that fact that the final outcome was better had world-changing…

I feel the main difference is cars can't compress time in the way an array of computers can. I could win this competition with an infinitely parallel array of random characters typed by infinite monkeys on infinite typewriters instantly since one of them would be perfectly right given infinite submissions. When I make my tweet I would pick a single monkey cus I need infinite money to feed my infinite workforce and that's more impressive clearly.

Now obviously it's more impressive as they don't have infinite compute and had finite time but the car only has one entry in each race unless we start getting into some anime ass shit with divergent timelines and one of the cars (and some lesser amount of horses) finishing instantly.

To your last point we don't know that this was cheaper since they don't disclose the cost. I would blindly guess a mechanical turk for the same cost would outperform at least today.

Re: DeepMind and OpenAI win gold at ICPC

#182

Post by gwbrooks »

in-silico wrote: post #161
furyofantares wrote: post #153

Earlier quoted context omitted.

Yeah man, and it would be wild to publish an article titled "Ford Mustang and Honda Civic win gold in the 100 meter dash at the Olympics" if what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run. Actually that's too generous, because the humans are given a time limit in ICPC, and there's no clear mapping to say how the LLM's compute should be limite…

Cars going faster than humans or horses isn't very interesting these days, but it was 100+ years ago when cars were first coming on the scene. We are at that point now with AI, so a more fitting headline analogy would be "In a world first, automobile finishes with gold-winning time in horse race". Headlines like those were a sign that cars would eventually replace horses in most use-cases, so the fact that we could b…

It was more than interesting 100+ years ago -- it was the subject of wildly inconsistent, often fear-based (or incumbent-industry-based) regulation.

A vetoed 1896 Pennsylvania law would have required drivers who encountered livestock to "disassemble the automobile" and "conceal the various components out of sight, behind nearby bushes until [the] equestrian or livestock is sufficiently pacified". The Locomotive on Highways Act of 1865 required early motorized vehicles to be preceded by a person on foot waving a red flag or carrying a red lantern and blowing a horn.

It might not quite look like that today, but wild-eyed, fear-based regulation as AI use grows is a real possibility. And at least some of it will likely seem just as silly in hindsight.

Re: DeepMind and OpenAI win gold at ICPC

#183

Post by Swizec »

dwohnitmok wrote: post #180
Swizec wrote: post #155

Earlier quoted context omitted.

> "did we build a vehicle faster than a horse, yes/no?" Yeah fair. There's also that famous human vs horse race that happens every few years. So far humans keep winning (because it's long distance)

If you're talking about the Man versus Horse Marathon ( https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon ) it's the other way around. Overwhelmingly the horses win. Only occasionally does the human.

I stand corrected. My memory garbled that. Thanks!

Re: DeepMind and OpenAI win gold at ICPC

#184

Post by hnfong »

furyofantares wrote: post #153
asboans wrote: post #104

Earlier quoted context omitted.

Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it. I…

Yeah man, and it would be wild to publish an article titled "Ford Mustang and Honda Civic win gold in the 100 meter dash at the Olympics" if what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run. Actually that's too generous, because the humans are given a time limit in ICPC, and there's no clear mapping to say how the LLM's compute should be limite…

All the while with skeptics snarkily commenting "Cars can move fast, but they can't really run like a human!"

Re: DeepMind and OpenAI win gold at ICPC

#185

Post by amluto »

OtherShrezzing wrote: post #147
amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

The human teams also get limited to one computer shared between 3 people. The models have access to an effectively unbounded number of computers. My argument does feel a bit like the “Watson doesn’t need to physically push the button” equivalents from when that system beat Jeopardy for the first time. I assume 5 hours on a single high-end Mac would probably still be enough compute in the near future.

I found the Watson match to be rather absurd. It would have been much more interesting if the rules had been modified so that all contestants had, say, two seconds two press the buzzer and that the contestant who got to answer first would be chosen by random selection among those who pressed the button. This would at least have made the competition be about who could come up with the most correct answers (questions).

Re: DeepMind and OpenAI win gold at ICPC

#186

Post by levocardia »

JohnKemeny wrote: post #8
NitpickLawyer wrote: post #2

So this year SotA models have gotten gold at IMO, IoI, ICPC and beat 9/10 humans in that atcoder thing that tested optimisation problems. Yet the most reposted headlines and rethoric is "wall this", "stangation that", "model regression", "winter", "bubble", doom etc.

There is a clear difference between what OpenAI manages to do with GPT-5 and what I manage to do with GPT-5. The other day I asked for code to generate a linear regression and it gave back a figure of some points and a line through it. If GPT-5, as claimed, is able to solve all problems in ICPC, please give the instructions on how I can reproduce it.

If you can't get a modern LLM to generate a simple linear regression I think what you have is a problem between the keyboard and the chair...

Re: DeepMind and OpenAI win gold at ICPC

#187

Post by hnfong »

ototot wrote: post #45
JohnKemeny wrote: post #26

Earlier quoted context omitted.

What makes you say that they are easier? Are there more people who manages to solve a problem from ICPC than from IOI? How do you compare those? There were at least 2 very simple problems in IOI this year. I haven't read the ICPC problem set, and perhaps there are some low-hanging fruits, but I highly doubt it.

Because I'm a ICPC medalist (not this year though) but not a IOI medalist. Another evidence is that you only have 5 hours to solve 3 problems in IOI, but you need to solve 10+ problems in ICPC. It's impossible to have all 10+ problems to at IOI level in ICPC.

Medals in both contests depend on your relative ranking (and of course depends on the difficulty of qualifying for them).

Doesn't say anything about the difficulty of the questions themselves though.

Re: DeepMind and OpenAI win gold at ICPC

#188

Post by levocardia »

raspasov wrote: post #158
paxys wrote: post #62

Earlier quoted context omitted.

My response simply is that performance in coding competitions such as ICPC is a very different skillset than what is required in a regular software engineering job. GPT-5 still cannot make sense of my company's legacy codebase even if asked to do the most basic tasks that a new grad out of college can figure out in a day or two. I recently asked it to fix a broken test (I had messed with it by changing one single ass…

This. Dealing with the problems of a real-world legacy code base is the exact opposite of a perfectly constrained problem, verified for internal consistency probably by computers and humans, of all things, and presented neatly in a single PDF. There are dozens, if not 100s, of assumptions that humans are going to make while solving a problem (i.e., make sure you don't crash the website on your first day at work!) tha…

>Waymo cars are still being supervised by human drivers nearly 100% of the time

That seems...highly implausible?

Re: DeepMind and OpenAI win gold at ICPC

#189

Post by jdietrich »

gwbrooks wrote: post #182
in-silico wrote: post #161

Earlier quoted context omitted.

Cars going faster than humans or horses isn't very interesting these days, but it was 100+ years ago when cars were first coming on the scene. We are at that point now with AI, so a more fitting headline analogy would be "In a world first, automobile finishes with gold-winning time in horse race". Headlines like those were a sign that cars would eventually replace horses in most use-cases, so the fact that we could b…

It was more than interesting 100+ years ago -- it was the subject of wildly inconsistent, often fear-based (or incumbent-industry-based) regulation. A vetoed 1896 Pennsylvania law would have required drivers who encountered livestock to "disassemble the automobile" and "conceal the various components out of sight, behind nearby bushes until [the] equestrian or livestock is sufficiently pacified". The Locomotive on Hi…

For more than thirty years, the speed limit for cars in Britain was 4mph - a self-propelled vehicle travelling faster than walking pace was obviously unconscionably dangerous.

To celebrate the raising of the speed limit to a daring 12mph, a group of motorists organised a drive from London to Brighton. At the time, driving 54 miles in a single day was seen as an audacious feat and few people imagined that such a great distance could be travelled in such complicated and newfangled contraptions without mechanical incident.

For decades, the car was seen as a plaything for the wealthy that served no practical purpose. The car only became an important mode of transportation after very many false starts and against strong opposition.

https://en.wikipedia.org/wiki/Locomotive_Acts#Locomotives_Ac...

https://en.wikipedia.org/wiki/London_to_Brighton_Veteran_Car...

Re: DeepMind and OpenAI win gold at ICPC

#190

Post by andoando »

roadside_picnic wrote: post #151
amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

> whereas the teams are allowed to bring a 25-page PDF This is where I see the biggest issue. LLMs are first-and-foremost text compression algorithms . They have a compressed version of a very good chunk of human writing. After being text compression engines, LLMs are really good at interpolating text based on the generalization induced by the lossy compression. What this result really tells us is that, given a reaso…

You can use the same framing for human reasoning except its over visual/auditority/spatial data and not just text.

You don't remember every detail of what you've seen correct? You store some lossy compression like "I went to a park"

Post reply on HN