someone posted an attempt at an open source implementation of alphagozero https://github.com/yhyu13/AlphaGOZero-python-tensorflow
anyone try it yet ?
Alpha Go Zero: How and Why It Works
31–40 of 116 posts
Re: Alpha Go Zero: How and Why It Works
#32The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a huge amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a g…
> These are the two biggest distinguishing characteristics of AlphaGo Zero compared to previous AlphaGos, and the OP doesn't discuss either of them.
David Silver disagrees. The most critical distinguishing characteristic is the expert/tree iteration which makes stable self-play possible at all.
Re: Alpha Go Zero: How and Why It Works
#33If the author's here: some of the math formulas don't render correctly. In particular, 10^170 is parsed as 10^{1}70, and $5478$ shows up without TeX applied to it.
Thanks, fixed those
Re: Alpha Go Zero: How and Why It Works
#34someone posted an attempt at an open source implementation of alphagozero https://github.com/yhyu13/AlphaGOZero-python-tensorflow anyone try it yet ?
There's also Leela Zero [1].
Re: Alpha Go Zero: How and Why It Works
#35I wonder how the STYLE of Alpha Go Zero is regarded by human experts. Is it far different from AlphaGo? Why bother learning from AlphaGo if they can learn from AlphaGo Zero? Did they unleash a second "Master" program? I am wondering if the "better" strategy moves are now super wacky and weird and break all theory.
You might be interested in the Redmon'd Review series on YouTube! Michael Redmond, 9P, and Chris Garlock just posted a short announcement that they'll be reviewing some Zero games. Michael mentioned that he'll focus on the opening in the next video.
Re: Alpha Go Zero: How and Why It Works
#36Go has been studied for hundreds of years. In many cases, by people who study the game since their childhood and work on it as a full-time occupation. The consequence of Alpha Go Zero is that it can, in a matter of days, disregard and surpass all human knowledge about the game. Maximizing a score margin has been equated for a long time with maximizing your probability of winning. Alpha Go doesn't play like that... it…
Even kyus like myself play more conservatively when winning. Alphago just takes this to the extreme but it is common on most levels of play (from single digit kyus to the best professional players).
Re: Alpha Go Zero: How and Why It Works
#37The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a huge amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a g…
could you elaborate on this point? what you're saying sounds like dynamic programming, which does not reduce the state space at all, just saves on redundant computations (and is a favourite of programming interviews everywhere)
Re: Alpha Go Zero: How and Why It Works
#38Mageek, any reason why they haven't applied this to chess yet?
Re: Alpha Go Zero: How and Why It Works
#39The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a huge amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a g…
Interesting, a TD algorithm, developed by a Canadian AI researcher now working with Deepmind in the early 1990s, was previously used to beat expert players at Backgammon and advanced human understanding of the game:
> TD-Lambda is a learning algorithm invented by Richard S. Sutton based on earlier work on temporal difference learning by Arthur Samuel. This algorithm was famously applied by Gerald Tesauro to create TD-Gammon, a program that learned to play the game of backgammon at the level of expert human players.
> TD-Gammon achieved a level of play just slightly below that of the top human backgammon players of the time. It explored strategies that humans had not pursued and led to advances in the theory of correct backgammon play.
Re: Alpha Go Zero: How and Why It Works
#40Can this technique be used to write a strong chess engine?
Yes. As described, all that’s needed is a way to imagine all possible moves from a game state, and check if a game state corresponds with win/tie/loss. That is possible in Tic-Tac-Toe, Chess, and Go.
Surprisingly, they haven't done it though. Wonder why. Is it because of the number of different state changes that can occur from a particular position? Maybe Go is easier to solve than chess.
It would be very interesting to see a computer improve on chess openings.