Why don't use the same approach for chess?
It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?
AlphaGo Zero: Learning from scratch
21–30 of 324 posts
Re: AlphaGo Zero: Learning from scratch
#22I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…
The core technique of AlphaGo is using tree search as a "policy improvement operator". Tree search doesn't work on most real-world tasks: the "game state" is too complex, there are too many choices, it's hard to predict the full effect of any choice you might make, and there often isn't even a "win" or "lose" state which would let you stop your self-play.
This version explicitly does not use tree search.
Re: AlphaGo Zero: Learning from scratch
#23> Previous versions of AlphaGo initially trained on thousands of human amateur and professional games to learn how to play Go. AlphaGo Zero skips this step and learns to play simply by playing games against itself, starting from completely random play. So technically this version has lost every game it's ever won. Jokes aside, it's pretty interesting to note that they were able to combine the "policy" and "value" net…
https://blog.openai.com/ I believe their DotA 2 AI uses that approach
Correct - we saw a similar phenomenon of rapid capability gain via self-play in our Dota 2 work: https://blog.openai.com/more-on-dota-2/
Re: AlphaGo Zero: Learning from scratch
#24"It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features." This is amazing! The technology they came up with must be super generic.
Super, super impressive work. I'd love to see how hard it is to apply the architecture to other games/problems that work well with self-play.
Re: AlphaGo Zero: Learning from scratch
#25Why don't use the same approach for chess? It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?
I think chess is less compelling because, in a sense, it is a "solved problem" - superhuman AI chess players already exist.
And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.
Re: AlphaGo Zero: Learning from scratch
#26Earlier quoted context omitted.
The core technique of AlphaGo is using tree search as a "policy improvement operator". Tree search doesn't work on most real-world tasks: the "game state" is too complex, there are too many choices, it's hard to predict the full effect of any choice you might make, and there often isn't even a "win" or "lose" state which would let you stop your self-play.
This version explicitly does not use tree search.
MCTS means "Monte-Carlo Tree Search". It's the core of the algorithm. The big difference is that it doesn't use rollouts, or random play: it chooses where to expand the tree based only on the neural network.
Re: AlphaGo Zero: Learning from scratch
#27Anyone happen to have a link (or can PM one to me) to a copy that isn't behind a paywall?
They already answered you but go to the bottom of the Nature page for a link to the game records (Supplementary Information).
Re: AlphaGo Zero: Learning from scratch
#28I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…
... yeah, before reading that link my position was "Wow, that's super neat, but Go is a pretty well-defined game," and after reading it I remembered that my position maybe a year or two ago was "Chess is a well-defined game that's beatable by AI techniques but Go is acknowledged to be much harder and require actual intelligence to play and won't be solved for a long while" and now I'm worried. Thanks for posting that.
Re: AlphaGo Zero: Learning from scratch
#29Earlier quoted context omitted.
This version explicitly does not use tree search.
MCTS means "Monte-Carlo Tree Search". It's the core of the algorithm. The big difference is that it doesn't use rollouts, or random play: it chooses where to expand the tree based only on the neural network.
No, 'habitue is correct. This new blog post says that the new software no longer does game readouts and just uses the neural net.
Re: AlphaGo Zero: Learning from scratch
#30Is this evidence of a broader leap forward in machine learning, or are these advancements domain-specific? In other words, could these innovations be applied to other fields and applications?
I think the fact that it's no longer using Monte Carlo tree search is a huge step forward in the generalizability of the technique. But go is still
- a perfect information game
- with a relatively small input size (vs. arbitrary computer vision)
- cheap to simulate
- discrete action space
- deterministic
This isn't to take away from the magnitude of the achievement, but the nature of the problem itself makes the result less applicable to many tasks we might want to use RL for.