11–20 of 84 posts
That does sound fishy,
Earlier quoted context omitted.
I agree, but: > Most board and card games fit this description. Most card games don't, because they are not perfect information games.
Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.
For Heads Up (two players) Limit (bet sizes are fixed) Hold Em (a popular modern variant with two hole cards and five community cards) such a strategy was developed and you can view it online, it's essentially unbeatable. Its name is Cepheus.
No, is an optimization of something already existing. An innovation, but not a breakthrough per se. Edit: this is an oversimplification
It could perhaps be said, then, that the Alpha series as a whole is a rolling breakthrough? Each new generation introduces changes that do not seem major compared to the ideas in the foundation of Deep Learning, but this actually allows to surpass state-of-the-art, improving performance on all relevant parameters (and not by a few percent). Dismissing these updates as technicalities is a dubious position.
Edit: https://github.com/glinscott/fishtest http://tests.stockfishchess.org/tests
The article glosses over why the 4 hours was possible. Firstly, a major challenge in training an AI of this sort is getting enough labelled data. They played 300,000 games from memory. Under normal circumstances, that requires access to 300,000 games played by experts so the AI can learn to copy what the export does. That is how Alpha Go did it. AlphaZero neatly side steps this by generating it's own training data by…
Self play reinforcement learning has been a thing for decades it really isn't new. That's how I wrote my Lines of Acton and Dominion AIs years ago. (Disclaimer: I had a sign flip bug in the Reinforcement Learning step of the Lines of Action bot that made it worse every generation! After a few thousand iterations it was _really_ bad at playing lines of action)
(Acton is a suburb of London and has more railway stations bearing its name than any other place in the UK)
The article glosses over why the 4 hours was possible. Firstly, a major challenge in training an AI of this sort is getting enough labelled data. They played 300,000 games from memory. Under normal circumstances, that requires access to 300,000 games played by experts so the AI can learn to copy what the export does. That is how Alpha Go did it. AlphaZero neatly side steps this by generating it's own training data by…
On the timeline issue of course, I don't it's reasonable to make any assumptions about what that implies about the algorithms. It may be that Chess was solved, the dev team went on a 6 month binge, then shoved a Go board in front of the machine and it learned it in a weekend. Alternatively they may have gotten to a point where 5 of their top target games had special cases to learn before they were ready to show off the Chess engine, and that it took 7 months to get the other applications over the line.
"However, the experimental setting does not seem fair. The version of Stockfish used was not the last one but, more importantly, it was run in its released version run on a normal PC, while AlphaZero was ran using considerable higher processing power. For example, in the TCEC competition engines play against each other using the same processor." That does sound fishy,
I'm actually not joking. I wonder how much different it would be to teach an AI like this how to play more complex games. I imagine Axis and Allies wouldn't take much, but Third Reich is notoriously complicated. The quickest war-length game I've played took a week of playing 3-4 hours per day and games like that seem to me to be much more similar to real world problems, with multiple different sorts of trade offs that interlock with each other.
Are neural AIs like this actually feasible to train for problems like that or are other AI techniques better suited to it? What about games with multiple different game systems, like board games with a card game element to them like Settlers of Catan? Would you need to use several different types of AI to optimize different parts of the game?
The article glosses over why the 4 hours was possible. Firstly, a major challenge in training an AI of this sort is getting enough labelled data. They played 300,000 games from memory. Under normal circumstances, that requires access to 300,000 games played by experts so the AI can learn to copy what the export does. That is how Alpha Go did it. AlphaZero neatly side steps this by generating it's own training data by…
Learning by self-play is nearly as old as AI itself. TD Gammon, one of the very first algorithms to reach superhuman levels in a nontrivial game, learned by self play. The basic ingredients for AlphaGo Zero, Monte-Carlo tree search and use of a convnet to evaluate board positions, were known already.
The major contribution of the AlphaGo Zero and AlphaZero was IMO the realisation that MCTS acts as a "policy improvement operatior", and that reinforcement learning becomes far more stable when it's used in conjunction with MCTS.
It's a major contribution and could represent a big shift in the field. But we won't be able to judge how big of a contribution until the research is more reproducable.
The article glosses over why the 4 hours was possible. Firstly, a major challenge in training an AI of this sort is getting enough labelled data. They played 300,000 games from memory. Under normal circumstances, that requires access to 300,000 games played by experts so the AI can learn to copy what the export does. That is how Alpha Go did it. AlphaZero neatly side steps this by generating it's own training data by…
It was done a long time ago. Arthur Samuel's chequers program (1959) improved by playing against itself in thousands of games.
People can and do improve that way too: https://www.chess.com/forum/view/general/playing-against-you...
"However, the experimental setting does not seem fair. The version of Stockfish used was not the last one but, more importantly, it was run in its released version run on a normal PC, while AlphaZero was ran using considerable higher processing power. For example, in the TCEC competition engines play against each other using the same processor." That does sound fishy,
It would be good to see a definitive playoff. I've no real doubt that AlphaZero would triumph, but in people's breathless coverage of the games nobody seems to point out Stockfish's various 5-10 pawn blunders (all of which my version of Stockfish finds when annotating the games with 60 seconds per move).