51–60 of 324 posts
Earlier quoted context omitted.
It does, during training.
Tree search is also used during play. In the paper, they pit the pure neural net against other versions of the algorithm -- it ends up slightly worse than the version that played Fan Hui, at about 3000 ELO.
I also wonder how much it’s style of play changes if it were re trained, due to the random start that it is given. Maybe that would produce something like seeds for procedurally generated worlds in games. Like if they could find a seed for a Chinese or japanese players, or ones that more aggressive styles. This is some pretty cool work and may open up even more doors for pure reinforcement learning
"It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features." This is amazing! The technology they came up with must be super generic.
But I suppose they still do the searching/pruning with a separate piece of code (not a neural network).
Genetic algorithms were used to evolve new, more efficient variants of existing electronic circuits. I dug it up - it was: https://www.scientificamerican.com/magazine/sa/2003/02-01/#a... Article "Evolving inventions". I have no idea if there is an open-access version anywhere.
As far as I remember, that approach led to some patents, because some of the inventions were better than existing solutions. One of the examples in the article was a low-pass filter (I dont remember if AI version was actually better or worse than human-made).
The essential element of this approach was that in electronics (as in go) there exist a well defined set of rules, that allows researchers to build a simulation engine with optimization/evaluation function that the AI targets by itself, without supervision. It's great to see that this approach is still alive, although in my humble opinion, application in electronics is much more interesting than Go.
Why don't use the same approach for chess? It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?
I think chess is less compelling because, in a sense, it is a "solved problem" - superhuman AI chess players already exist. And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.
But shogi is much more obscure outside of Japan than go or chess, so it gets less interest, especially in the large-board variants.
Is this evidence of a broader leap forward in machine learning, or are these advancements domain-specific? In other words, could these innovations be applied to other fields and applications?
I think the fact that it's no longer using Monte Carlo tree search is a huge step forward in the generalizability of the technique. But go is still - a perfect information game - with a relatively small input size (vs. arbitrary computer vision) - cheap to simulate - discrete action space - deterministic This isn't to take away from the magnitude of the achievement, but the nature of the problem itself makes the resu…
> Previous versions of AlphaGo initially trained on thousands of human amateur and professional games to learn how to play Go. AlphaGo Zero skips this step and learns to play simply by playing games against itself, starting from completely random play. So technically this version has lost every game it's ever won. Jokes aside, it's pretty interesting to note that they were able to combine the "policy" and "value" net…
> So technically this version has lost every game it's ever won. No, they've also played it against AlphaGo Lee and AlphaGo Master. The SGFs are available at: https://www.nature.com/nature/journal/v550/n7676/extref/natu...
I remember reading ages ago in Scientific American about a much more interesting (and useful) AI application of this technique. Genetic algorithms were used to evolve new, more efficient variants of existing electronic circuits. I dug it up - it was: https://www.scientificamerican.com/magazine/sa/2003/02-01/#a... Article "Evolving inventions". I have no idea if there is an open-access version anywhere. As far as I re…
The other SA article on this was The Darwin Chip which I think went into more detail.
One of the limitations was the lack of documentation for the actual bitstream.