Would love to see some gameplay!
AlphaGo Zero: Learning from scratch
61–70 of 324 posts
Re: AlphaGo Zero: Learning from scratch
#62Earlier quoted context omitted.
I think chess is less compelling because, in a sense, it is a "solved problem" - superhuman AI chess players already exist. And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.
Well, I'd love to see NN solution beating top chess engines. It might also introduce novelty to the game, just as regular engines did
It'd be particularly useful to have a chess bot that can play badly in the same way a human does.
The problem with the current chess bots is that they play badly, badly. They choose a terrible random mistake to make every few moves, while some of their other moves are brilliant. They cannot accurately mimic beginner or intermediate level players.
Re: AlphaGo Zero: Learning from scratch
#63Earlier quoted context omitted.
The core technique of AlphaGo is using tree search as a "policy improvement operator". Tree search doesn't work on most real-world tasks: the "game state" is too complex, there are too many choices, it's hard to predict the full effect of any choice you might make, and there often isn't even a "win" or "lose" state which would let you stop your self-play.
This version explicitly does not use tree search.
If you read the paper, they do in fact still use monte-Carlo tree search. They just simplify their usage in conjunction with reducing the number of neural networks to 1
Re: AlphaGo Zero: Learning from scratch
#64Why don't use the same approach for chess? It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?
Denis Hassabis was asked this in a Q&A after a talk he gave and according to him someone did this (bootstrap a chess engine from self play) successfully, while still being a student and was hired by them subsequently.
I didn't see the talk, but I'm guessing he was referring to the Giraffe engine done by Matthew Lai (https://arxiv.org/abs/1509.01549). The main thing there is that he only learns an evaluation function, not a policy. Giraffe still uses classical alpha-beta search over the full action space. AFAIK nobody has learned a decent policy network for chess, probably because 1) it's super tactical, and 2) nobody cares that much because alpha-beta is so strong
Re: AlphaGo Zero: Learning from scratch
#65Earlier quoted context omitted.
MCTS means "Monte-Carlo Tree Search". It's the core of the algorithm. The big difference is that it doesn't use rollouts, or random play: it chooses where to expand the tree based only on the neural network.
No, 'habitue is correct. This new blog post says that the new software no longer does game readouts and just uses the neural net.
Read the white paper. MCTS is still involved, right the way through.
Re: AlphaGo Zero: Learning from scratch
#66Earlier quoted context omitted.
I think chess is less compelling because, in a sense, it is a "solved problem" - superhuman AI chess players already exist. And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.
Shogi is probably the closest historical game in terms of complexity to Go. Some of the larger variants might exceed Go's complexity if played with drops, though that's not normally done. And Go played on a 9x9 board (like standard Shogi) has a substantially lower state space complexity (and almost certainly lower by other measures as well.) But shogi is much more obscure outside of Japan than go or chess, so it gets…
[deleted]
Re: AlphaGo Zero: Learning from scratch
#67Absolutely amazing. I'm dying to see the game records!
Spoiler, they're in the paper: https://deepmind.com/documents/119/agz_unformatted_nature.pd...
Re: AlphaGo Zero: Learning from scratch
#68Earlier quoted context omitted.
Well, I'd love to see NN solution beating top chess engines. It might also introduce novelty to the game, just as regular engines did
It'd be particularly useful to have a chess bot that can play badly in the same way a human does. The problem with the current chess bots is that they play badly, badly. They choose a terrible random mistake to make every few moves, while some of their other moves are brilliant. They cannot accurately mimic beginner or intermediate level players.
In a sense, that would be like replicating the human brain's functionality, including the bugs and limitations.
Re: AlphaGo Zero: Learning from scratch
#69Anyone know why they trained on TPUs? My understanding is that the main benefit of a TPU is inference. Is this not true?
Re: AlphaGo Zero: Learning from scratch
#70[deleted]