Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

181–190 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#181

Post by muraiki »

sanxiyn wrote: post #87
mikekchar wrote: post #77

Earlier quoted context omitted.

I'm curious to see if "San Gatsu no Lion" (the Lion of March) will spark interest. I highly recommend it to anyone interested in more slice-of-life/drama kinds of things. It's quite a beautiful anime/manga, even if the shogi isn't quite centre stage.

Recommendation seconded, Sangatsu no Lion is a lovely work. On the other hand, it has been running for 10 years (!), if it could spark interest like Hikaru no Go, it would have happened already.

Shion no Ou is another good shogi anime. I haven't read the manga.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#182

Post by sanxiyn »

hmate9 wrote: post #162

So AlphaGo Zero used 4 TPUs while AlphaZero used 1500. It’s not immediately obvious to me why there is this massive difference. Can anyone elaborate?

Both used 4 TPUs at playing time. At training time, AlphaGo Zero used unspecified amount of computing resource, AlphaZero used 5000 TPUs for self-play.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#183

Post by [deleted] »

make3 wrote: post #164
Houshalter wrote: post #21

Earlier quoted context omitted.

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

they are a huge company (Google) with access to top top top talent (experts) and infinite hardware resources. I don't know why it would be surprising if they acheived performance that hadn't been acheived before

[deleted]

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#184

Post by robrenaud »

thom wrote: post #171
bhouston wrote: post #170

Earlier quoted context omitted.

I think that it is almost always possible to extract optimized models from nn and implement them faster. I wonder if this can be generalized. Nn to optimized fix algo for Max speed? This has to already exist as it is very obvious.

I dunno, seems like Google would just do this instead of keep around the pesky neural net at runtime. There's an _awful_ lot of computation going on inside, and it's necessarily hugely interconnected. I'd be impressed if someone had already done it, but it seems a great avenue of research if not. I suppose it goes hand in hand with models for which you can actually _explain_ their results, which certainly is an activ…

There are well-known techniques that work pretty well to shrink neural nets a lot while keeping almost all of their performance. See Geoffrey Hinton's model distillation papers.

The first AlphaGo paper had a system that used tons of computation, and was followed up by one that used much less and worked even better. Not speaking for Google, but I think it's a bit of a race to publish great results first. I wouldn't be surprised to see something better than this that uses 1000 times less resources published in a year or two, just like what happened with Go. First prove it's possible, than figure out how to make it much more efficient.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#186

Post by nl »

arcticfox wrote: post #172
nl wrote: post #101

Earlier quoted context omitted.

The number of positions evaluated is the number evaluated. Speed doesn’t change that. Speed probably made the initial self play training quicker though.

Can't those teraflops be applied to evaluating more positions instead of deciding which positions to evaluate? It seems that the metric should be compute time, not positions evaluated.

Presumably they both had equal clock time - that is a standard chess rule so it would be surprising to see it different.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#187

Post by CydeWeys »

Retric wrote: post #178
CydeWeys wrote: post #143

Earlier quoted context omitted.

TPUs aren't "cheating" though, as they can be used for generalized machine learning models, and not just Go. Computer graphics is still an impressive achievement even when done on a GPU instead of a CPU.

I think his point is if you devote X Flops to something then a fair comparison would be to also give X Flops to the competitor. The specifics of how an algorithm does not matter as much as the total resources used and outcome.

A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operation, but dollars do matter.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#188

Post by thom »

robrenaud wrote: post #184
thom wrote: post #171

Earlier quoted context omitted.

I dunno, seems like Google would just do this instead of keep around the pesky neural net at runtime. There's an _awful_ lot of computation going on inside, and it's necessarily hugely interconnected. I'd be impressed if someone had already done it, but it seems a great avenue of research if not. I suppose it goes hand in hand with models for which you can actually _explain_ their results, which certainly is an activ…

There are well-known techniques that work pretty well to shrink neural nets a lot while keeping almost all of their performance. See Geoffrey Hinton's model distillation papers. The first AlphaGo paper had a system that used tons of computation, and was followed up by one that used much less and worked even better. Not speaking for Google, but I think it's a bit of a race to publish great results first. I wouldn't be…

Thanks for these googlable hints. :)

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#189

Post by jmmcd »

gallerdude wrote: post #3

I wonder if being an expert at one game makes it easier to be an expert at another. If so, then maybe the examples are datasets, and convergence would be able to complete new tasks after a few examples.

Well, it's not doing anything like that for now. Even though the algorithm, in an abstract sense, is the same for all three games, in fact it's a new network for each of the three games, with architecture and input features adapted to the game, and then trained from scratch.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#190

Post by tantalor »

gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

> doesn't work as well in chess: it works better

That's not how "as well" works.

Post reply on HN