Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

201–210 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#201

Post by gwern »

cdancette wrote: post #105
grumpopotamus wrote: post #55

Earlier quoted context omitted.

>it only takes 4 hours of training to beat Stockfish In that time I figure they used the equivalent of about 1000 cpu-years. Imagine the things we'll be able to achieve as we can do more and more computation in less and less time.

But how many "cpu hours" of human work were used to design stockfish? You can't really compare that. Some scientist say the brain has a power of several petaflops, so if you use this, I guess the design of stockfish was way less efficient. You can't really compare things to cpu years, it doesn't make sense. Power consumption would be a better metric I think.

A ton of CPU has gone into Stockfish, if only for their distributed computing project fishnet: http://tests.stockfishchess.org/tests

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#202

Post by sireat »

bluecalm wrote: post #158
computerphage wrote: post #42

Earlier quoted context omitted.

It beat the winner of TCEC-2016, Stockfish, with a record of 28-72-0. That's zero losses.

If I run SF on my desktop computer it will kill SF run on my phone. It doesn't prove anything. Comparing TPUs and CPUs is hard but they could've at least let SF run on what is considered top of the line setup and sensible settings (1GB hash memory is very limited, 8GB is standard for rapid games on a quad core CPU, let alone 64core one).

I can't figure out the reason for this stingy 1GB hash memory limit when using 64 cores. It pretty much negates advantage of 64 cores vs say 4/6 cores.

A nefarious suggestion would be that setting 1GB limit ensures that Alpha would always have the edge in depth as Stockfish would be forced to prune long lines to preserve hash memory.

Maybe someone who has read Stockfish source code can comment how Stockfish prunes hash memory.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#203

Post by yters »

nopinsight wrote: post #50
maxander wrote: post #24

Earlier quoted context omitted.

Meanwhile a human player considers But unsettlingly few, nonetheless.

Humans are also much weaker than AlphaZero in these three games. The difference in the numbers of positions searched might be responsible for a substantial part of that.

It'd be interesting to weaken AZ until it is on par with a human, and then compare moves evaluated. I'd suspect humans still evaluate significantly fewer moves.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#204

Post by sillysaurus3 »

soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

https://i.imgur.com/kwCyiHn.png

That was a bad move for white to play. It's easy to win when your opponent throws the game.

No human player would trade queens in that situation.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#206

Post by varelse »

allenz wrote: post #64
gwern wrote: post #30

Earlier quoted context omitted.

> So in a way, the real triumph of the AlphaGo series was the TPU and GPU army. Eh. It's still searching many fewer positions than Stockfish is.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days.

Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network during MCTS and 64 (second generation) TPUs to train this thing according to the methods. That's a nice mix of using INT8 for inference and FP16ish for training IMO.

In contrast, I personally own 8 GTX Titan XP class GPUs and 8 more GTX Titan XM GPUs across 4 desktops in my home network. I'd love to experiment with algorithms like this, but I suspect I'd get just about nowhere due to insufficient sampling. These algorithms are insanely inefficient at sampling at the beginning. So I guess I will seed the network with expert training data to see if that speeds things up.

That said, more brilliant work from David Silver's group! But not all of us have 5,000 TPUs/GPUs just sitting around so there's still a lot more work/research to make this more accessible to less sexy problems.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#207

Post by varelse »

Retric wrote: post #198
CydeWeys wrote: post #187

Earlier quoted context omitted.

A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operati…

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

I think we need to start capping total electricity and total $$$. I'd love to see AlphaZero 20W pitted against that other 20W supercomputer. When humans fall to that, be afraid(tm).

I'll even be charitable in order to simulate the existence of school/teachers/books: training from the start gets 2KW. But gameplay still gets capped to 20W.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#208

Post by tdb7893 »

dandermotj wrote: post #140
Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

This is definitely a scientific paper. Pretty much no scientific paper comes with source code and the majority of scientific papers are not reproducible without an entire university department of resources anyway.

My main thing about source code and scientific papers is that it would just be so easy to release the source code along with the paper. Even if people don't reproduce work source code would often help to understand it as often I'm a little unclear on implementation details, which source code would be able to greatly clarify.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#209

Post by CydeWeys »

Retric wrote: post #198
CydeWeys wrote: post #187

Earlier quoted context omitted.

A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operati…

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

This argument is about state-of-the-art chess, not chess as a mobile phone game. Humans are so bad at chess compared to the best programs now that even a smartphone app can't be defeated by people.

Also, mobile phones have Internet access, so there's no reason the algorithm has to run on the phone itself. It could run on TPUs in the cloud. It's common for many games to have server-side components. Though this isn't even necessary except maybe if Magnus Carlsen wants to play it.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#210

Post by CydeWeys »

varelse wrote: post #207
Retric wrote: post #198

Earlier quoted context omitted.

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

I think we need to start capping total electricity and total $$$. I'd love to see AlphaZero 20W pitted against that other 20W supercomputer. When humans fall to that, be afraid(tm). I'll even be charitable in order to simulate the existence of school/teachers/books: training from the start gets 2KW. But gameplay still gets capped to 20W.

Electricity isn't free though; why can't it simply be rolled into cost? Just assign it a standard cost per kW-hr and charge accordingly. This more accurately reflects economic incentives driving hardware development.
Post reply on HN