Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
1–10 of 282 posts
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#2Certainly a significant achievement. Also, kind of interesting that the AlphaGo team spent a lot of energy to convince us Go is much harder than Chess, only to turn around and tell us that it is amazing that it can also win at Chess.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#3I wonder if being an expert at one game makes it easier to be an expert at another. If so, then maybe the examples are datasets, and convergence would be able to complete new tasks after a few examples.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#4I'm not sure it's really fair to compare Stockfish to AlphaZero; AlphaZero used 24h of 5000 TPUs in compute time, and still needed 4 TPUs in real play, while Stockfish ran on just 64 threads and 1GB RAM. Nonetheless, still an impressive achievement.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#5If you have seen the Stockfish project you will see many hardcoded weights in the configuration, found through experimentation. All these adjustments took probably years to achieve... and now Alpha Go Zero just self-learns everything and surpasses it.
Would be good to see Deepmind's solution play Arimaa and Stratego, and see what kind of strategy it comes up with. Or weird variations of Go.
Eventually this tech will make it into military strategy simulators and that's where things will get really messed up. 4 star generals will be replaced by bots.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#6Two things to note:
1) Alpha Zero beats AlphaGo Zero and AlphaGo Lee and starts tabla rasa
2) "Shogi is a significantly harder game, in terms of computational complexity, than chess (2, 14): it is played on a larger board, and any captured opponent piece changes sides and may subsequently be dropped anywhere on the board. The strongest shogi programs, such as Computer Shogi Association (CSA) world-champion Elmo, have only recently defeated human champions (5)"
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#7Very happy to see this result. It's like a moral victory for humans, as alphago is more human like (discounting montecarlo search) than stockfish. Maybe deep learning will give us the next Euler, Newton, or Einstein.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#8Certainly a significant achievement. Also, kind of interesting that the AlphaGo team spent a lot of energy to convince us Go is much harder than Chess, only to turn around and tell us that it is amazing that it can also win at Chess.
> only to turn around and tell us that it is amazing that it can also win at Chess.
What they're demoing here is a single, general formula for mastering multiple games. Start with empty AG0, then teach it chess from scratch until it is the strongest player on the planet.
Go back to an empty slate, with the same exactly "untrained" AG0, and now teach it Go, to the same result. No fine-tuning for the domain of the game you are training -- it is general(ized).
That's the gist I'm getting from this.
question for someone who has time to read the paper: can you train it to master chess and go at the same time? or is it one or the other? I'm assuming the latter.
edit: check out the graph on the 4th page. AlphaZero, which can master chess and shogi, can beat AlphaGo Zero, the implementation specifically designed for Go, at its own game.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#9"We also analysed the relative performance of AlphaZero’s MCTS search compared to the state-of-the-art alpha-beta search engines used by Stockfish and Elmo. AlphaZero searches just 80 thousand positions per second in chess and 40 thousand in shogi, compared to 70 million for Stockfish and 35 million for Elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more selectively on the most promising variations – arguably a more “human-like” approach to search, as originally proposed by Shannon." <- Amazing!
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#10This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only takes 4 hours of training to beat Stockfish. This is going to be an impetus for researchers to explore solving many more MDPs than just chess or Go using expert iteration... ("There is no fire alarm.")