Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

261–270 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#261

Post by EvgeniyZh »

varelse wrote: post #206
allenz wrote: post #64

Earlier quoted context omitted.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

P100 is 20 FP16 TFLOPs, V100 is ~30. So 4 TPU gen 2 is ~9 P100 or 6 V100

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#262

Post by Retric »

CydeWeys wrote: post #258
Retric wrote: post #250

Earlier quoted context omitted.

That's not why this is relevant. Given equivalent hardware it's still a worse solution for chess . The value is you can get results of similar quality with vastly more compute power even without 1,000+ years of analysis. Otherwise the only takeaway is this failed to improve the state of the art.

"Equivalent hardware" is only relevant if we're talking about cost . When measured by that metric, the TPUs are indeed superior. Raw operations is an irrelevant metric given the existence of economic purpose-specific hardware that can perform a lot more of the operations required for matrix multiplication than for general computation. GPUs work exactly the same.

Again, cost is relative to hardware you have. If you own a supercomputer already and you want to run chess on it for whatever reason it matters what the performance you get from each algorithm on that hardware. If your going to buy new hardware it's design depends on performance across every algorithm you expect to use.

So, the only case where chess performance per $ matters is if you are only going to ever use that hardware to run chess. In every other case which is the vast majority of the time you care about diffent metrics.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#263

Post by empath75 »

Retric wrote: post #198
CydeWeys wrote: post #187

Earlier quoted context omitted.

A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operati…

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

Why? Just do the computation in the cloud.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#264

Post by nojvek »

skybrian wrote: post #225
gwern wrote: post #30

Earlier quoted context omitted.

> So in a way, the real triumph of the AlphaGo series was the TPU and GPU army. Eh. It's still searching many fewer positions than Stockfish is.

Unlike in most algorithms where correctness and performance are independent, chess engines can't be evaluated without testing performance at the same time; faster is not just faster, it changes the results. So there is a tradeoff between the depth of the search and quality of evaluation. For traditional chess algorithms, better evaluation was rarely worth the cost; it would slow down the search so much that it didn't…

Agree with this. Stockfish is fast enough to run on modern iphone and Android phones. AlphaZero most probably not.

But the fact that a generic algorithm absolutely destroys humans and their human crafted programs is the most interesting.

Yes TPU + GPU army is a huge amount of computation power but I'm sure they'll be research coming out trying to compress the algorithms enough to use the same computation power as stock fish.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#265

Post by skc »

I'm only a fairly pedestrian chess player, but I looked at one of these games between AGZ and SF and aside from the endgame, AGZ played in a manner that almost seemed alien. It seemed to completely ignore various little rules of thumb which is to be expected in hindsight but fairly mind-blowing when you actually watch a game.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#266

Post by nojvek »

CydeWeys wrote: post #237
varelse wrote: post #224

Earlier quoted context omitted.

Sure, but the whole point of the above idea is to compare our 20W computers to what we can build that eats 20W. And don't give Silicon Valley ideas about disrupting the lucrative Mechanical Turk ecosystem by scaling it up with ideas borrowed from growing veal because some VC sociopath will take it seriously. Just sayin'...

And I'm saying that this 20W limitation isn't particularly meaningful, as many organizations have way more power at their disposal to throw at a problem than that. The economics of a given solution, on the other hand, is applicable at all scales.

Meaningful in the sense that if an AI plays against humans, is it smarter at the same energy efficiency of humans.

We are comparing machine intelligence vs human intelligence.

It can be said that with more computational power, you can raise intelligence. Human brains consume the most power relative to body size than any other animal.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#267

Post by nojvek »

varelse wrote: post #206
allenz wrote: post #64

Earlier quoted context omitted.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

It's definitely worth a shot reproducing the results.

On the other hand, Google will make a shit load of money when they make TPUs available on gcloud. Papers like this are great marketing for them.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#268

Post by nojvek »

evandijk70 wrote: post #111
Houshalter wrote: post #21

Earlier quoted context omitted.

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

Look at Figure 2, and remember that DM has access to a lot of hardware. At short thinking times, AlphaZero is weaker than Stockfish. This is equivalent to longer thinking times with weaker hardware, and it is likely that the former applications of NNs to chess had hardware that was a 1000-fold slower than what DM has access to. This means that even if the approach was identical to DMs, they would not have seen a bett…

In essence MCTS + NN is just another way of tree search just like AlphaBeta or its brute force cousin Minimax.

AlphaZero just tries to be smarter about which branches to evaluate so it can go deeper.

But I would love to see AlphaZero (trained) run side by side with stockfish on an iPhone hardware and defeat it. That would be a more apples to apples comparison.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#269

Post by nojvek »

gwern wrote: post #240
gwern wrote: post #201

Earlier quoted context omitted.

A ton of CPU has gone into Stockfish, if only for their distributed computing project fishnet: http://tests.stockfishchess.org/tests

To be a little more precise: Stockfish has used >5,667,382 CPU-hours (5.6 million CPU-hours) adding up just the participants who contributed >10,000 CPU-hours according to https://github.com/mcostalba/Stockfish/blob/master/Top%20CPU...

Yes but that's training time. At runtime AlphaZero got magnitudes more computation power than stockfish.

A more fair comparison would be use as much computation needed for training but for runtime, use equal wattage hardware.

E.g 20W of cpu in mobile phone running stockfish to 20W of GPU in nvidia TX2 running AlphaZero to 20W human brain.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#270

Post by erk__ »

317070 wrote: post #212
Aissen wrote: post #165

Earlier quoted context omitted.

Comparing projects done in the open with multiple different universities, on public funds, with something done behind closed doors with only personnel from a commercial entity is pretty far-fetched.

Why? How are any of the factors you mention related to verifiability? How does being supported by public funds with academic personnel from multiple universities make LIGO any more verifiable for me at home? At least I can run these games against my stockfish, thus verifying the result. The method I cannot verify, but being able to verify the results is already more than most of science.

You can maybe not do the experiments from Cern, but you can do the calculations most of the software they use for that is open source iirc.
Post reply on HN