Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

221–230 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#221

Post by saagarjha »

ototot wrote: post #202
512 wrote: post #139

Earlier quoted context omitted.

Are you an ICPC World Finals medalist? Because winning an IOI bronze medal is _way_ easier than even qualifying for the ICPC WF, and less than 10% of the teams at the WF get medals. I'd go as far as saying that gold at the IOI is probably easier than getting an ICPC medal. (One is individual and the other is in teams, but my point stands).

Yes. I'm a ICPC World Final medalist. Totally agree that IOI bronze is way easier than ICPC bronze. In terms of rank/ratio, ICPC medals are more like IOI gold. I stated things like that because I thought it's a bit easier to let people know the difficulty difference. (Agree weirdly though)

Did you win the Putnam, though?

Re: DeepMind and OpenAI win gold at ICPC

#222

Post by chpatrick »

raspasov wrote: post #167
chpatrick wrote: post #152

Earlier quoted context omitted.

It doesn't say anywhere that Gemini used any of those things at ICPC, or that it used more real-world time than the humans. Also, who cares? It's a self contained non-human system that could solve an ICPC problem it hasn't seen before on its own, which hasn't been achieved before. If there was a savant human contestant with photographic memory who could remember every previous ICPC problem verbatim and can think real…

I think "hasn't seen before" is a bit of an overstatement. Sure, the problem is new in the literal sense that it does exist verbatim elsewhere, but arguably, any competition problem is hardly novel: they are all some permutation of problems that exist and have been solved before: pathfinding, optimization, etc. I don't think anyone is pretending to break new scientific ground in 5 hours.

It's not new scientific ground but a machine beating a challenging computer science problem unassisted is a big deal. If they can do that then there are a lot of other challenging things they can do.

Re: DeepMind and OpenAI win gold at ICPC

#223

Post by Darmani »

OtherShrezzing wrote: post #147
amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

The human teams also get limited to one computer shared between 3 people. The models have access to an effectively unbounded number of computers. My argument does feel a bit like the “Watson doesn’t need to physically push the button” equivalents from when that system beat Jeopardy for the first time. I assume 5 hours on a single high-end Mac would probably still be enough compute in the near future.

The people saying that were wrong, BTW -- Watson did have to physically press a button.

https://www.wired.com/2011/02/ibm-watson-speed/

> __Brown: __ Watson has a mechanical button-presser. It uses the same signaling device [the button] that the human competitors use in the game. Once Watson has decided that it wants to ring in because it has found an answer with a high-enough confidence, and it receives the signal that the buzzers are open and you can ring in, it then has to trigger the mechanical button presser and mechanically press the button.

Re: DeepMind and OpenAI win gold at ICPC

#224

Post by smokel »

NoahZuniga wrote: post #143
smokel wrote: post #53

The best thing of the ICPC is the first C, which stands for "collegiate". It means that you get to solve a set of problems with three persons, but with only one computer. This means that you have to be smart about who is going to spend time coding, thinking, or debugging. The time pressure is intense, and it really is a team sport. It's also extra fun if one of the team members prefers a Dvorak keyboard layout and vi…

Actually collegiate means that the contestants are in college.

Ha, shows what I know :) As a non-native speaker I had always assumed it referred to working with colleagues. Etymologically related, but apparently not the same.

Re: DeepMind and OpenAI win gold at ICPC

#225

Post by theywillnvrknw »

amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

The more important question I'd like answered is did the models have the same number of attempts (submissions) as human contestants.

If not were penalities applied for multiple solutions.

Re: DeepMind and OpenAI win gold at ICPC

#226

Post by OtherShrezzing »

Darmani wrote: post #223
OtherShrezzing wrote: post #147

Earlier quoted context omitted.

The human teams also get limited to one computer shared between 3 people. The models have access to an effectively unbounded number of computers. My argument does feel a bit like the “Watson doesn’t need to physically push the button” equivalents from when that system beat Jeopardy for the first time. I assume 5 hours on a single high-end Mac would probably still be enough compute in the near future.

The people saying that were wrong, BTW -- Watson did have to physically press a button. https://www.wired.com/2011/02/ibm-watson-speed/ > __Brown: __ Watson has a mechanical button-presser. It uses the same signaling device [the button] that the human competitors use in the game. Once Watson has decided that it wants to ring in because it has found an answer with a high-enough confidence, and it receives the signal t…

Watson originally just sent an electronic signal. The physical button pushing was introduced later, to stave off the criticism that it had an unfair advantage.

Re: DeepMind and OpenAI win gold at ICPC

#227

Post by LunaSea »

matheusd wrote: post #220
LunaSea wrote: post #212

Earlier quoted context omitted.

Power is one thing, efficiency is another. Humans are more efficient watt for watt than any AI ever invented. Now if you were to limit AIs to 400 watts we could probably thinks it's fair.

> Humans are more efficient watt for watt than any AI ever invented. Indeed they are. For now. The long term trend is not in our favor.

I disagree, the long term trend is that we do not have the available electricity and water for this to continue for much longer.

Re: DeepMind and OpenAI win gold at ICPC

#228

Post by guyomes »

krisoft wrote: post #165
furyofantares wrote: post #153

Earlier quoted context omitted.

Yeah man, and it would be wild to publish an article titled "Ford Mustang and Honda Civic win gold in the 100 meter dash at the Olympics" if what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run. Actually that's too generous, because the humans are given a time limit in ICPC, and there's no clear mapping to say how the LLM's compute should be limite…

> what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run That would be indeed an interesting race around the time cars were invented. Today that would be silly, since everyone knows what cars are capable of, but back then one can imagine a lot more skepticism. Just as there is a ton of skepticism today of what LLMs can achieve. A competition like thi…

> under the same conditions

That's a very interesting question. When comparing wildly different computing machines, how to make a fair comparison?

At least two criteria comes in mind: the volume and the energy consumption.

Indeed we can safely assume that more volume and more energy leads to more computation power. For example, it is not fair to compare a 10m^3 room filled with computers with 10cm^3 computer. The same goes with the number of kilowhat-hours used.

Thinking further on those two criteria for GPUs and humans, we could also consider the access to energy and volume. First, energy access for machines has dramatically increased since the industrial revolution. Second, volume access for machines has also increased since the beginning of the mass production. In particular, creating one cube meter of new GPUs is faster than giving birth to a new human.

tldr: fair comparison of two machines should take into account their volume and their energy consumption. On the other hand, this might be mitigated by how fast a machine can increase its volume, and what is its bandwidth for energy consumption.

Re: DeepMind and OpenAI win gold at ICPC

#229

Post by vrighter »

theragra wrote: post #102
amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

I think your analogy is lacking. Human brain is much more efficient, so it is not right to say "giving a human team a week instead of five hours". Most likely, the whole OpenAI compute cannot match one brain in terms of connections and relations and computation power.

so in that case, don't compare them to humans in the first place

Re: DeepMind and OpenAI win gold at ICPC

#230

Post by vrighter »

dist-epoch wrote: post #81
amluto wrote: post #59

I've contemplated this a bit, and I think I have a bit of an unconventional take: First, this is really impressive. Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are co…

If you want to play that game, let's compute how much energy was spent to grow, house and educate one team since they were born, over 20 years against how much was spent training the model.

a human being uses about 1 gWh in a lifetime on average.

Training one model nowadays uses more than that. So the model also cost more to train than to raise a team member (possibly all three of them)

Post reply on HN