Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

111–120 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#111

Post by evandijk70 »

Houshalter wrote: post #21
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

Look at Figure 2, and remember that DM has access to a lot of hardware. At short thinking times, AlphaZero is weaker than Stockfish. This is equivalent to longer thinking times with weaker hardware, and it is likely that the former applications of NNs to chess had hardware that was a 1000-fold slower than what DM has access to. This means that even if the approach was identical to DMs, they would not have seen a better performance of NNs than the classical alpha beta approach.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#113

Post by narrator »

So when are they going to apply this to Atari Games or well anything? The next step is they have one AI figure out the rules by making a GAN that imitates player behavior and the other AI be Alpha Go which tweaks the GAN inputs to generate different moves to win. Voila...Almost General Purpose AI that can learn to play any game.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#114

Post by [deleted] »

ericand wrote: post #6

Two things to note: 1) Alpha Zero beats AlphaGo Zero and AlphaGo Lee and starts tabla rasa 2) "Shogi is a significantly harder game, in terms of computational complexity, than chess (2, 14): it is played on a larger board, and any captured opponent piece changes sides and may subsequently be dropped anywhere on the board. The strongest shogi programs, such as Computer Shogi Association (CSA) world-champion Elmo, have…

[deleted]

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#115

Post by kaffeemitsahne »

pacaro wrote: post #38
ericand wrote: post #6

Two things to note: 1) Alpha Zero beats AlphaGo Zero and AlphaGo Lee and starts tabla rasa 2) "Shogi is a significantly harder game, in terms of computational complexity, than chess (2, 14): it is played on a larger board, and any captured opponent piece changes sides and may subsequently be dropped anywhere on the board. The strongest shogi programs, such as Computer Shogi Association (CSA) world-champion Elmo, have…

Shogi is a fun game, it always feels a little sad that it doesn't get more exposure outside of Japan (and my understanding is that, by and large, in Japan it is considered an "old persons" game) Because captured pieces change sides, there is less of an "endgame" scenario, and as a beginner (like me) it is very easy to put too many captured pieces back into play, which makes it hard to defend everything and essentiall…

I've been interested in learning both shogi and xiangqi for a while. If anyone knows a nice engine with graphical frontend for either game, I'd love to know. Wasn't able to find much the last time I looked.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#116

Post by thom »

zwischenzug wrote: post #107
thom wrote: post #98

Earlier quoted context omitted.

Back to zero at 41 ply... I shan't give a running commentary anymore.

It's fascinating isn't it? I'd love to see this vs Magnus.

Pretty sure I input the moves wrong now I'm looking at it. Humans really have no place in chess. :P

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#117

Post by gcp »

xoroshiro wrote: post #108
Cybiote wrote: post #69

Earlier quoted context omitted.

Let's break this down and consider things carefully. To informed researchers, what is most surprising here is not that the AlphaGo Zero algorithm beat stockfish but that MCTS managed to outperform Alpha-beta search. I'll venture a hypothesis as to why this was. Informed skepticism would have discounted MCTS against alpha-beta search but wouldn't have put much stock into the idea that Neural Networks couldn't learn be…

Probably the wrong engine to test this with then. Although it's interesting nonetheless. It's pretty well known that chess engines have this trade-off between searching and evaluating. Among the consistent top 3 I suppose Stockfish is the easiest to test, being open source and all. It's pretty well regarded that Komodo has the best evaluation function though. Even if it doesn't keep up with the nodes/sec of Houdini a…

Your comments about Stockfish, Komodo etc are entirely subjective. "It's pretty well regarded". No it's not.

You can't disconnect the search part from the discussion, as the search selectivity is ALSO learned by the neural network.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#118

Post by 110011 »

What an amazing result! Evaluating fewer (by a factor of 1000) positions AlphaZero still beats Stockfish.

In the figure on its preferred openings I find it very interesting that it doesn't like the Ruy Lopez very much over training time (there is a small bump but that is transient). I am hardly a chess expert but I know that it was very favored at the world championships so maybe the chess world will be turned upside down by this result now?

Positing that the chess world is bigger than the Go world (in terms of interest and finances) there is probably going to be a race to replicate these results "at home" and train yourself before your competitors :)

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#119

Post by proaralyst »

nl wrote: post #104
zwischenzug wrote: post #61

I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…

The edits on this comment make it the second best HN comment of all time.

What's the first?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#120

Post by [deleted] »

Cybiote wrote: post #109
roenxi wrote: post #49

Earlier quoted context omitted.

Is it really though? A big class of imperfect information games can be modeled by having a record of everything the agent has seen so far. Then it has exactly the same, if not more, information available than a human player in the same position. We know that with equal information AIs can make better decisions than humans (see also, AlphaGo :] ) so at that point the AI could reasonably be expected to achieve superhum…

The difficulty of imperfect information is from cross cutting through information sets and partial observability. With perfect information games like chess or Go, one can solve subgames with guarantees that the equilibrium is the same as for the full game. This is not the case for games like poker, which is why they have been difficult. In addition to that, for n > 2 players, there are no longer theoretical guarantee…

[deleted]
Post reply on HN