Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

21–30 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#21

Post by Houshalter »

gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#23

Post by gwern »

Houshalter wrote: post #21
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

It's certainly not the first NN chess program. You may remember one of OP author's Giraffe NN (https://arxiv.org/abs/1509.01549) which was essentially 'AlphaGo for chess'. But like the original AG, it struggles to learn and Lai had a lot less computation as a student than he does now at DM. What they're doing is applying AlphaGo Zero expert iteration with some simplifications and TPUs. And that pwns previous work like Giraffe the way AlphaGo Zero pwns AlphaGo. Quantity becomes a quality all its own.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#24

Post by maxander »

magoghm wrote: post #9

"We also analysed the relative performance of AlphaZero’s MCTS search compared to the state-of-the-art alpha-beta search engines used by Stockfish and Elmo. AlphaZero searches just 80 thousand positions per second in chess and 40 thousand in shogi, compared to 70 million for Stockfish and 35 million for Elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more…

Meanwhile a human player considers But unsettlingly few, nonetheless.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#25

Post by xianshou »

One impressive statistic from the paper: AlphaZero analyzes 80,000 chess positions per second, while Stockfish looks at 70,000,000. Seventy million, three orders of magnitude higher. Yet AG0 beats Stockfish half the time as White and never loses with either color.

A stunning demonstration of generality indeed.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#26

Post by jtolmar »

firebones wrote: post #13

A lot of the graphs in the paper seem to level out as they hit the level of the opponent. It makes me wonder to what extent AlphaGo Zero is merely optimizing to beat flaws in existing opponents' current implementations (even if "existing opponents" == all available opponents' data and algorithms today) rather than generalizable insights into the underlying game. Because wouldn't you expect that unless we are at the t…

ELO ratings level out eventually for a given pool of opponents. If a player already wins every game against all available opponents, there's no evidence that can tell you if they suddenly got twice as good.

If tracking improvements past the state of the art is important I think they'd have to freeze the algorithm every 400 ELO or so and rate the improved versions against the last snapshot.

(Doesn't really apply to the stockfish case, but it does to the other two games.)

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#27

Post by deepnotderp »

gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

See the thing is though, Giraffe's evaluation actually was better than Stockfish's evaluation function, but it took much longer, and thus wasn't able to search as deep as Stockfish et al. So in a way, the real triumph of the AlphaGo series was the TPU and GPU army.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#29

Post by Sukotto »

Is this a library or something I can download and try training myself (on a small scale)?

I'm not in a position to read the paper right now, so my apologies if that's covered in there. I want to ask just in case it's not, while this is still on the front page.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#30

Post by gwern »

deepnotderp wrote: post #27
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

See the thing is though, Giraffe's evaluation actually was better than Stockfish's evaluation function, but it took much longer, and thus wasn't able to search as deep as Stockfish et al. So in a way, the real triumph of the AlphaGo series was the TPU and GPU army.

> So in a way, the real triumph of the AlphaGo series was the TPU and GPU army.

Eh. It's still searching many fewer positions than Stockfish is.

Post reply on HN