For those complaining about the TPU resources used during self training it is worth noting that Stockfish has used over 10,000 CPU hours for tuning its parameters. See https://github.com/mcostalba/Stockfish/blob/master/Top%20CPU...
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
241–250 of 282 posts
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#242Earlier quoted context omitted.
It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.
But better evaluation gives you asymptotic speedups. You can give Stockfish several times its computation (which is already a lot, I mean, 64 threads, come on) and it doesn't make good use of it since it just runs into the search wall. If you gave Stockfish the equivalent in CPU power (and I'm not sure this is a fair hypothetical since part of the appeal of NNs is that they have such efficient hardware implementation…
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#243Earlier quoted context omitted.
hmm...13.Nce5 looks like the move no strong human would play, and I suspect even engines after going sufficiently deep wouldn't choose it (I haven't checked it though).
My perspective as FIDE master who has played Ruy Lopez Exchange type of positions for 30+ years. 9. Qe1 is a pretty normal maneuvering move 13. Ncxe5??! looks like a major howler. Ask 100 strong chess players and 99 of them would completely ignore it. You are giving up a piece for two pawns in an open position and black has no real weaknesses. There is no real basis for a sacrifice. This shouldn't work. The crazy thi…
I agree Ncxe5 looks crazy, but the weirder thing to me is that Stockfish offers a repetition the very next move. So it can't be caused by having high contempt (favouring wins over draws).
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#244The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#245Earlier quoted context omitted.
Is it really though? A big class of imperfect information games can be modeled by having a record of everything the agent has seen so far. Then it has exactly the same, if not more, information available than a human player in the same position. We know that with equal information AIs can make better decisions than humans (see also, AlphaGo :] ) so at that point the AI could reasonably be expected to achieve superhum…
The difficulty of imperfect information is from cross cutting through information sets and partial observability. With perfect information games like chess or Go, one can solve subgames with guarantees that the equilibrium is the same as for the full game. This is not the case for games like poker, which is why they have been difficult. In addition to that, for n > 2 players, there are no longer theoretical guarantee…
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#246Earlier quoted context omitted.
>it only takes 4 hours of training to beat Stockfish In that time I figure they used the equivalent of about 1000 cpu-years. Imagine the things we'll be able to achieve as we can do more and more computation in less and less time.
> In that time I figure they used the equivalent of about 1000 cpu-years. Are you using some kind of conversion factor from TPUs to CPUs? If so, what is it? And is it valid to do that? You could convert the amount of time it took to render an hour's worth of gameplay from 1 GPU-hour to 50 CPU-days (or whatever), but is that really meaningful?
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#247The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…
https://i.imgur.com/kwCyiHn.png That was a bad move for white to play. It's easy to win when your opponent throws the game. No human player would trade queens in that situation.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#248Earlier quoted context omitted.
It's easy for many to think that solving Go and chess means we can also solve household work like cleaning, cooking and washing dishes but it's actually harder.
Next up: Google's Deepmind AI learns to perform arithmetic tabula rasa . More seriously, it seems Deepmind and the AI community in general is having a Streetlight effect problem, i.e. looking for AI in what works now, rather than coming to terms with the hard challenges. This explains why there are so many papers on GANs. People are just doubling down on what works (where the streetlight is), rather than acknowledgin…
But most people in the research community already know how amazing it would be to make an affordable household robot or a search-and-rescue robot or a self-driving car. Many labs (including mine) are working on it. The streetlight adds a small bias, but the bigger problem is that we have no idea how to build human-level AI.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#249Earlier quoted context omitted.
Right right, but my comparison was between giraffe and AlphaGo , not neural networks and Stockfish.
But it looks like AlphaGo is searching fewer positions per second than Giraffe did. AlphaZero evaluates 80K positions per second, according to this paper, and the Giraffe paper says that Giraffe averaged 258570 evaluations per second when running STS. While we can't directly compare the computer power, this implies that AZ has learned a better representation.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#250Earlier quoted context omitted.
I think you misunderstood. Sure, if you are willing to deal with the increased costs and lowered reliability you could write a chess program that required massive server resources. But, I don't think a lot of people would pay for that vs. having a program that just runs on there phone and still beats them. So, in practice without a significant subscription fee you are going to be limited to cellphone hardware. PS: In…
Again, this entire article and discussion is about state-of-the-art chess. As in, literally working to "solve" the game and develop optimal strategy. I don't understand what relevance casual mobile chess games have. Computer chess is already very far beyond human capabilities, and it can't be pressed further just using mobile phone hardware (nor is that a reasonable restriction). It'd be like in a discussion about Sp…
Otherwise the only takeaway is this failed to improve the state of the art.