Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

231–240 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#231

Post by varelse »

skybrian wrote: post #227
varelse wrote: post #206

Earlier quoted context omitted.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

I wonder how much the hardware would cost to rent for a researcher not working at Google?

So 3 2nd generation TPUs are ~= 1 Volta class GPU ~= $3 per hour on-demand on AWS: https://aws.amazon.com/ec2/pricing/on-demand/ and ~$1 (75 cents at the moment with p3.8xlarge and its 4 GPUs) in spot: https://aws.amazon.com/ec2/spot/pricing/ if you take the time to build a robust framework.

And to make things simple, let's do it all in FP16 because INT8 on Volta ~= 1/2 a first generation TPU, but FP16 ~= 3 first generation TPUs at INT8 (sad, right?), an accident that occurred because P100 didn't support INT8, but consumer variants did.

So, 5,064/3 = 1,688 Volta GPUs ~= $5000 per hour, probably half that reserved, a quarter of that in spot.

Say you need a week to train this, so $200K-$800K...

You can buy DGX-1Vs off-label for about $75K. Say they costs $20K annually to host. Say you use them for 3 years, so total TCO is ~135K, which comes down to $0.64/hour.

Conclusion: p3.8xl spot instances are currently a steal! But I don't have ~$200K burning a hole in my pocket, so I guess I'm out of luck.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#232

Post by forgot-my-pw »

sireat wrote: post #157
parimarjan wrote: post #128

Earlier quoted context omitted.

hmm...13.Nce5 looks like the move no strong human would play, and I suspect even engines after going sufficiently deep wouldn't choose it (I haven't checked it though).

My perspective as FIDE master who has played Ruy Lopez Exchange type of positions for 30+ years. 9. Qe1 is a pretty normal maneuvering move 13. Ncxe5??! looks like a major howler. Ask 100 strong chess players and 99 of them would completely ignore it. You are giving up a piece for two pawns in an open position and black has no real weaknesses. There is no real basis for a sacrifice. This shouldn't work. The crazy thi…

Thanks for the analysis

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#233

Post by forgot-my-pw »

Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

There's a project currently that emulates AlphaGo Zero using crowdsourcing: https://github.com/gcp/leela-zero . You can run it on the browser too and it will submit the games after: https://ntt123.github.io/leela-zero/

Hope such a chess project like this will be available in the future.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#234

Post by Retric »

CydeWeys wrote: post #209
Retric wrote: post #198

Earlier quoted context omitted.

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

This argument is about state-of-the-art chess, not chess as a mobile phone game. Humans are so bad at chess compared to the best programs now that even a smartphone app can't be defeated by people. Also, mobile phones have Internet access, so there's no reason the algorithm has to run on the phone itself. It could run on TPUs in the cloud. It's common for many games to have server-side components. Though this isn't e…

I think you misunderstood. Sure, if you are willing to deal with the increased costs and lowered reliability you could write a chess program that required massive server resources.

But, I don't think a lot of people would pay for that vs. having a program that just runs on there phone and still beats them. So, in practice without a significant subscription fee you are going to be limited to cellphone hardware.

PS: In practice most games take about as much computing power from a server as a chat app as companies need to pay for that hardware. Remember 1,000,000+ X get's big unless you keep X very low.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#235

Post by Retric »

modeless wrote: post #223
Retric wrote: post #198

Earlier quoted context omitted.

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

Several phones already have neural net acceleration hardware in them today, including the latest iPhones.

Some iPhones are manufactured with this, but again if you have paid for the hardware you care about performance on that hardware. If you have yet to buy anything then theoretical performance per $ becomes the meaningful metric.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#236

Post by CydeWeys »

Retric wrote: post #234
CydeWeys wrote: post #209

Earlier quoted context omitted.

This argument is about state-of-the-art chess, not chess as a mobile phone game. Humans are so bad at chess compared to the best programs now that even a smartphone app can't be defeated by people. Also, mobile phones have Internet access, so there's no reason the algorithm has to run on the phone itself. It could run on TPUs in the cloud. It's common for many games to have server-side components. Though this isn't e…

I think you misunderstood. Sure, if you are willing to deal with the increased costs and lowered reliability you could write a chess program that required massive server resources. But, I don't think a lot of people would pay for that vs. having a program that just runs on there phone and still beats them. So, in practice without a significant subscription fee you are going to be limited to cellphone hardware. PS: In…

Again, this entire article and discussion is about state-of-the-art chess. As in, literally working to "solve" the game and develop optimal strategy. I don't understand what relevance casual mobile chess games have. Computer chess is already very far beyond human capabilities, and it can't be pressed further just using mobile phone hardware (nor is that a reasonable restriction).

It'd be like in a discussion about SpaceX's BFR designs to colonize Mars, someone comes in and questions why they're using retropropulsion since the requisite control systems are infeasibly expensive for amateur model rockets. It's a completely different discussion.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#237

Post by CydeWeys »

varelse wrote: post #224
CydeWeys wrote: post #222

Earlier quoted context omitted.

I don't think you can, and such a comparison is not really needed here anyway. People are not chattel slaves and cannot be racked into data centers to solve boring problems. Of course, you can hire people, and that has a well-defined cost, so it does all come down to money again.

Sure, but the whole point of the above idea is to compare our 20W computers to what we can build that eats 20W. And don't give Silicon Valley ideas about disrupting the lucrative Mechanical Turk ecosystem by scaling it up with ideas borrowed from growing veal because some VC sociopath will take it seriously. Just sayin'...

And I'm saying that this 20W limitation isn't particularly meaningful, as many organizations have way more power at their disposal to throw at a problem than that. The economics of a given solution, on the other hand, is applicable at all scales.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#239

Post by galkk »

sireat wrote: post #202
bluecalm wrote: post #158

Earlier quoted context omitted.

If I run SF on my desktop computer it will kill SF run on my phone. It doesn't prove anything. Comparing TPUs and CPUs is hard but they could've at least let SF run on what is considered top of the line setup and sensible settings (1GB hash memory is very limited, 8GB is standard for rapid games on a quad core CPU, let alone 64core one).

I can't figure out the reason for this stingy 1GB hash memory limit when using 64 cores. It pretty much negates advantage of 64 cores vs say 4/6 cores. A nefarious suggestion would be that setting 1GB limit ensures that Alpha would always have the edge in depth as Stockfish would be forced to prune long lines to preserve hash memory. Maybe someone who has read Stockfish source code can comment how Stockfish prunes ha…

Well, the one explanation is that they wanted to win "convincingly", thus 1m per move and so low memory amount for hash.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#240

Post by gwern »

gwern wrote: post #201
cdancette wrote: post #105

Earlier quoted context omitted.

But how many "cpu hours" of human work were used to design stockfish? You can't really compare that. Some scientist say the brain has a power of several petaflops, so if you use this, I guess the design of stockfish was way less efficient. You can't really compare things to cpu years, it doesn't make sense. Power consumption would be a better metric I think.

A ton of CPU has gone into Stockfish, if only for their distributed computing project fishnet: http://tests.stockfishchess.org/tests

To be a little more precise: Stockfish has used >5,667,382 CPU-hours (5.6 million CPU-hours) adding up just the participants who contributed >10,000 CPU-hours according to https://github.com/mcostalba/Stockfish/blob/master/Top%20CPU...
Post reply on HN