While this sounds impressive, I'll believe it when AlphaZero wins TCEC.
How can we fairly evaluate TPU engines vs. CPU engines?
Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
151–160 of 282 posts
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#152Did Magnus play against this? Is there a way we can see the game?
No he didn't play it. As far as I know, computers are already far ahead of humans in chess, so a further progress in this wouldn't really make a difference.
It would be interesting in one way though:
Magnus says he hates playing against computers, because "it's like being beaten by an idiot". Modern chess engines still make moves that are somewhat strategically weak, but they make up for it with amazing tactics.
It would be interesting to hear if Magnus thought AlphaZero played less like an idiot.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#153Discussion at the Computer Chess Club (CCC) forum:
http://www.talkchess.com/forum/viewtopic.php?topic_view=thre...
and
http://www.talkchess.com/forum/viewtopic.php?topic_view=thre...
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#154Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.
There are too many details missing for the results to be reproducible. Does it even qualify as a tech demo if the result only exists in DeepMind's lab?
This raises an interesting concept. If you cannot reproduce an experiment because of lack of resources, can you believe it? Or is this the equivalent of 'photoshopping your results'?
A similar problem exists in cosmology. Can you verify the multiverse model if you only have one universe to experiment in?
As data storage requirements in RAM and TPU power requirements increase to run certain models/algorithms, machine learning is becoming more obscure. Not only can we not understand how an AI is reaching its conclusions (inscrutability), we cannot even probe it (by tweaking parameters, etc) to find weak points (inaccesibility). This is actually a good thing. Where humans cannot tread, there can be no evil?
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#155Earlier quoted context omitted.
Back when AlphaGo was playing Lee Sedol I was thinking about a chess playing version in TCEC. The interesting thing is TCEC assumes a bit about the structure of the chess program. That is, the TCEC win-adjudication rule says that if both programs agree that one program is 6.5 pawns ahead for 8 turns in a row, they judge that program to be the winner. But programs like Alpha don't have an evaluation function that oper…
You can convert winning percentages to centipawns, so that's not a problem.
Could you explain your proposed conversion process?
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#156Earlier quoted context omitted.
The number of positions evaluated is the number evaluated. Speed doesn’t change that. Speed probably made the initial self play training quicker though.
Compute absolutely matters. With tree search, there's a tradeoff between scoring cost and positions evaluated. AlphaZero can evaluate fewer positions because it uses a huge amount of compute to accurately score each position. It's not just training. Training used 5,000 TPUs.
[deleted]
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#157I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…
hmm...13.Nce5 looks like the move no strong human would play, and I suspect even engines after going sufficiently deep wouldn't choose it (I haven't checked it though).
My perspective as FIDE master who has played Ruy Lopez Exchange type of positions for 30+ years.
9. Qe1 is a pretty normal maneuvering move
13. Ncxe5??! looks like a major howler.
Ask 100 strong chess players and 99 of them would completely ignore it. You are giving up a piece for two pawns in an open position and black has no real weaknesses. There is no real basis for a sacrifice.
This shouldn't work. The crazy thing is that Stockfish almost makes it work.
It is the kind of move you play when you absolutely must win and must win now.
The only reason Stockfish considered it is because of white pawn on a5 giving additional tactics in breaking up black pawn chain with a6 a couple of moves down. With pawn on a4 Ncxe5 wouldnt be worth attempting.
The crazy thing is that being such a bully almost worked!
At move 28. White looks very solid, with 3 perfect pawns for the piece + black has horrible weaknesses. 29. g3 is a bit suspect but the next super computer move is
31. Qxc7 this has to be losing but it is a typical computer bully move.
Most strong human players would prefer to defend h3 hole with Kg2 (on Qh5 f5 looks fine).
The idea is that black's white square bishop is boxed in with white pawns.
There must be a concrete reason why Stockfish did not play Kg2.
Overall the impression one gets is of very "human" play by Alpha and ultra aggressive play by Stockfish.
EDIT: so extremely impressive play by Alpha but a bit suspicious aggression by Stockfish.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#158While this sounds impressive, I'll believe it when AlphaZero wins TCEC.
It beat the winner of TCEC-2016, Stockfish, with a record of 28-72-0. That's zero losses.
If I run SF on my desktop computer it will kill SF run on my phone. It doesn't prove anything.
Comparing TPUs and CPUs is hard but they could've at least let SF run on what is considered top of the line setup and sensible settings (1GB hash memory is very limited, 8GB is standard for rapid games on a quad core CPU, let alone 64core one).
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#159It doesn't seem to like the Sicilian Defense (1.e4 c5), which is the most popular opening by human players. I wonder if this will change opening theory?
It looks as if it doesn't play 1.e4 much as white. Since these statistics are for self-play games, that means it won't get a lot of opportunities to play 1.e4 c5 as black. Still, it does seem as if it likes the Ruy Lopez and French better as black than it does the Sicilian. (It would be nice to see a little opening "tree" with move probabilities, rather than this list of 12 most-popular-among-humans openings.)
[EDITED to add:] A couple of other remarks:
Playing against Stockfish, the Sicilian seems to give it more wins as white and more losses as black than any of the other openings listed here.
What's shown here are two particular versions of the Sicilian; for all we know there's a lot more 1.e4 c5 in its self-play than the graphs suggest (e.g., maybe as white it prefers 2.c3 or 2.Nc3 or something). Eyeballing those graphs, these 12 openings account for substantially less than half of AlphaZero's self-play games.
Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning
#160Earlier quoted context omitted.
You can convert winning percentages to centipawns, so that's not a problem.
Could you explain your proposed conversion process?
Houdini for example tries to make it so that +1.00 evaluation is a win in 75% of cases in blitz games and +1.5 represents 90% chance of winning (http://www.cruxis.com/chess/houdini.htm). Anyway, this is not a problem at all, this was introduced so less electricity is wasted when the position is a clear win/loss.