41–50 of 84 posts
Earlier quoted context omitted.
> AlphaZero neatly side steps this by generating it's own training data by playing itself. If how to do this was "obvious", it would have been done a long time ago. Learning by self-play is nearly as old as AI itself. TD Gammon, one of the very first algorithms to reach superhuman levels in a nontrivial game, learned by self play. The basic ingredients for AlphaGo Zero, Monte-Carlo tree search and use of a convnet to…
its own training data
Earlier quoted context omitted.
Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.
Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. So that's hugely different. This truth is why strong poker analysts are always…
Instead of the output of the neural network being the move to make then, couldn't you make the output be the weight/probability you should fold, call or raise?
Earlier quoted context omitted.
What do you mean by "various 5-10 pawn blunders"? Is that a count of the number of blunders or some sort of score?
Stockfish and other mainstream engines will judge a position in units of centipawns - one hundredth of a pawn. Largely this will be to do with material on either side. For example, if you throw away a pawn with all else being equal, I'd be up 100 centipawns. If I'm white this gets written as "+1.00", and if I'm black it's "-1.00". The score is also based on a strategic evaluation of the position (and positions to com…
Can anyone summarise how self play works here if AlphaZero only starts out being told the rules of the game? Does it initially plays games using completely random moves as both players? Is it only told who the winner is with no other feedback? How is it able to learn e.g. that certain moves at the start eventually lead to a win?
When working in academia, I found it very common for research papers to not come with source code or enough information to allow you to replicate experiments yourself. You usually have to pester the author. I don't find the (valid) criticisms here that unusual. I'm not sure why they wouldn't release the moves for all the test games played though seeing is that should be simple to do.
As for Stockfish and AlphaZero running on different hardware...AlphaZero's approach is built around taking full advantage of what TPUs can do quickly and Stockfish doesn't utilise TPUs so how are you meant to make this fair? Does Stockfish eventually level out when you throw enough hardware at it? Doesn't DeepMind's claim about AlphaZero evaluating significantly less moves per turn invalidate the criticism about the hardware used?
Can anyone summarise how self play works here if AlphaZero only starts out being told the rules of the game? Does it initially plays games using completely random moves as both players? Is it only told who the winner is with no other feedback? How is it able to learn e.g. that certain moves at the start eventually lead to a win?
I suppose, yes. This is called "reinforcement learning".
Earlier quoted context omitted.
> People can and do improve that way too When learning Chess or Go, playing whole games with myself would be too tedious. Do people really do this? But I have played countless opening sequences in chess and many many joseki lines in go, I have practised various piece mating combos in chess and different yose techniques in go, one does chess problems from newspapers or books and of course everyone knows you do go prob…
> When learning Chess or Go, playing whole games with myself would be too tedious. Do people really do this? Chess World Champions Robert Fischer and Magnus Carlsen notoriously did as children. Magnus Carlsen even recommended it as one of his top 13 tips to improve : https://www.youtube.com/watch?v=FMaaHd7aFIs
Can anyone summarise how self play works here if AlphaZero only starts out being told the rules of the game? Does it initially plays games using completely random moves as both players? Is it only told who the winner is with no other feedback? How is it able to learn e.g. that certain moves at the start eventually lead to a win?