Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

221–230 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#221

Post by defertoreptar »

gamegoblin wrote: post #155
gcp wrote: post #131

Earlier quoted context omitted.

You can convert winning percentages to centipawns, so that's not a problem.

Could you explain your proposed conversion process?

Here's a relevant section from Deepmind's paper:

> We also measured the head-to-head performance of AlphaZero against each baseline player. Settings were chosen to correspond with computer chess tournament conditions: each player was allowed 1 minute per move, resignation was enabled for all players (-900 centipawns for 10 consecutive moves for Stockfish and Elmo, 5% winrate for AlphaZero). Pondering was disabled for all players.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#222

Post by CydeWeys »

varelse wrote: post #213
CydeWeys wrote: post #210

Earlier quoted context omitted.

Electricity isn't free though; why can't it simply be rolled into cost? Just assign it a standard cost per kW-hr and charge accordingly. This more accurately reflects economic incentives driving hardware development.

Sure, why not, but then how do we compare to a human?

I don't think you can, and such a comparison is not really needed here anyway. People are not chattel slaves and cannot be racked into data centers to solve boring problems.

Of course, you can hire people, and that has a well-defined cost, so it does all come down to money again.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#223

Post by modeless »

Retric wrote: post #198
CydeWeys wrote: post #187

Earlier quoted context omitted.

A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operati…

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

Several phones already have neural net acceleration hardware in them today, including the latest iPhones.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#224

Post by varelse »

CydeWeys wrote: post #222
varelse wrote: post #213

Earlier quoted context omitted.

Sure, why not, but then how do we compare to a human?

I don't think you can, and such a comparison is not really needed here anyway. People are not chattel slaves and cannot be racked into data centers to solve boring problems. Of course, you can hire people, and that has a well-defined cost, so it does all come down to money again.

Sure, but the whole point of the above idea is to compare our 20W computers to what we can build that eats 20W. And don't give Silicon Valley ideas about disrupting the lucrative Mechanical Turk ecosystem by scaling it up with ideas borrowed from growing veal because some VC sociopath will take it seriously. Just sayin'...

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#225

Post by skybrian »

gwern wrote: post #30
deepnotderp wrote: post #27

Earlier quoted context omitted.

See the thing is though, Giraffe's evaluation actually was better than Stockfish's evaluation function, but it took much longer, and thus wasn't able to search as deep as Stockfish et al. So in a way, the real triumph of the AlphaGo series was the TPU and GPU army.

> So in a way, the real triumph of the AlphaGo series was the TPU and GPU army. Eh. It's still searching many fewer positions than Stockfish is.

Unlike in most algorithms where correctness and performance are independent, chess engines can't be evaluated without testing performance at the same time; faster is not just faster, it changes the results.

So there is a tradeoff between the depth of the search and quality of evaluation. For traditional chess algorithms, better evaluation was rarely worth the cost; it would slow down the search so much that it didn't pay for itself.

But this performance tradeoff (like all optimizations) critically depends on hardware. Change the hardware and you change which optimizations are "worth it".

AlphaZero is clearly good at using TPU's to maximum effect. But what would its performance be in a CPU only environment? Maybe dumb but deeper searches still win there? This evaluation hasn't been done.

This isn't to say that the AlphaZero evaluation is "unfair". Rather that chess engines evolved to be too dependent on their environment. Getting maximum use out of CPU's is a strength, but not being able to use TPU's or even GPU's is a weakness.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#226

Post by namelost »

vinchuco wrote: post #136
namelost wrote: post #53

Earlier quoted context omitted.

They didn't demonstrate that AlphaGo Zero can beat Stockfish in a fair contest: i.e. take the amount of money they spent on Stockfish's CPU and RAM, buy a commodity GPU for AlphaGo and then see.

https://news.ycombinator.com/item?id=2232163

I'm sorry, I thought we were discussing the paper.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#227

Post by skybrian »

varelse wrote: post #206
allenz wrote: post #64

Earlier quoted context omitted.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

I wonder how much the hardware would cost to rent for a researcher not working at Google?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#228

Post by voxl »

Aissen wrote: post #166
dandermotj wrote: post #140

Earlier quoted context omitted.

This is definitely a scientific paper. Pretty much no scientific paper comes with source code and the majority of scientific papers are not reproducible without an entire university department of resources anyway.

> Pretty much no scientific paper comes with source code Are we blindly accepting this as science now ?

Yes? Sorry you've been out of the loop so long but science doesn't cater to your idealistic ideas of what it ought to be.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#229

Post by eternalcode »

EvgeniyZh wrote: post #148
1024core wrote: post #51

Earlier quoted context omitted.

So ... what if you combined Stockfish and AG0, and let AG0 explore 70M positions instead of 80K? Would it improve even faster?

What if you combined a bus that gets you to work in 10 minutes and plane that gets you from Paris to Brazil, would it get you from Paris to Brazil in 10 minutes?

yes. In an imaginary and hypothetical sense. :)

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#230

Post by skybrian »

backpropaganda wrote: post #137
visarga wrote: post #133

Earlier quoted context omitted.

It's easy for many to think that solving Go and chess means we can also solve household work like cleaning, cooking and washing dishes but it's actually harder.

Next up: Google's Deepmind AI learns to perform arithmetic tabula rasa . More seriously, it seems Deepmind and the AI community in general is having a Streetlight effect problem, i.e. looking for AI in what works now, rather than coming to terms with the hard challenges. This explains why there are so many papers on GANs. People are just doubling down on what works (where the streetlight is), rather than acknowledgin…

Yes, but it doesn't seem like much of a problem? Exploiting a breakthrough before moving on to harder problems isn't cheating, it's the smart thing to do. It might even turn out to be the fastest way to make progress on the harder problems.
Post reply on HN