Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

211–220 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#211

Post by cdelsolar »

I wanted to contact the authors directly but can't seem to find contact info at the moment, with a question. I hope some of you might know enough to answer it.

I'm interested in applying this method, or a similar neural-network / tabula rasa based method to the game of Scrabble. I read the original AlphaGo Zero paper and they mentioned that this method works best for games of perfect information. The standard Scrabble AI right now is quite good and can definitely beat top experts close to 50% of the time, but it uses simple Monte Carlo simulations to evaluate positions and just picks the ones that perform better. It doesn't quite account for defensive considerations or other subtleties of the game. I was wondering if anyone who had more insight into MCTS and NN would be able to talk me through how to apply this to Scrabble, or if it even makes sense. One of the issues I can see currently would be very slow convergence; as it has a luck factor, the algorithm could make occasional terrible moves and still win games, and thus be "wrongly trained".

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#212

Post by 317070 »

Aissen wrote: post #165
317070 wrote: post #134

Earlier quoted context omitted.

How do you replicate CERN experiments? The LHC? Hubble? LIGO? LISA? At least this paper is reproducible by people who have the compute, and many universities have super computers. Even at home, you can verify the results by replaying the games against stockfish. You might not be able to replicate the setup at home, but that does not mean it is not science.

Comparing projects done in the open with multiple different universities, on public funds, with something done behind closed doors with only personnel from a commercial entity is pretty far-fetched.

Why? How are any of the factors you mention related to verifiability? How does being supported by public funds with academic personnel from multiple universities make LIGO any more verifiable for me at home? At least I can run these games against my stockfish, thus verifying the result. The method I cannot verify, but being able to verify the results is already more than most of science.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#213

Post by varelse »

CydeWeys wrote: post #210
varelse wrote: post #207

Earlier quoted context omitted.

I think we need to start capping total electricity and total $$$. I'd love to see AlphaZero 20W pitted against that other 20W supercomputer. When humans fall to that, be afraid(tm). I'll even be charitable in order to simulate the existence of school/teachers/books: training from the start gets 2KW. But gameplay still gets capped to 20W.

Electricity isn't free though; why can't it simply be rolled into cost? Just assign it a standard cost per kW-hr and charge accordingly. This more accurately reflects economic incentives driving hardware development.

Sure, why not, but then how do we compare to a human?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#214

Post by sillysaurus3 »

soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

Uhh... These games are actually broken. From the second link: https://imgur.com/a/P5tG6

See for yourself:

https://lichess.org/Zqwn4Gzk#87

https://lichess.org/Zqwn4Gzk#88

EDIT: Nope, I'm just a noob.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#215

Post by rditooait »

sillysaurus3 wrote: post #214
soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

Uhh... These games are actually broken. From the second link: https://imgur.com/a/P5tG6 See for yourself: https://lichess.org/Zqwn4Gzk#87 https://lichess.org/Zqwn4Gzk#88 EDIT: Nope, I'm just a noob.

https://en.wikipedia.org/wiki/En_passant

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#216

Post by magoghm »

elcapitan wrote: post #139
thanatropism wrote: post #132

Earlier quoted context omitted.

The keyword is "reinforcement learning".

I know the names of the general concepts, I was wondering if someone has concrete recommendations on where to start and which books/frameworks are sort of beginner-friendly.

You can check this Reinforcement Learning Course by David Silver on YouTube: https://www.youtube.com/watch?v=2pWv7GOvuf0&t=836s

By the way, I believe David Silver was the lead programmer for AlphaZero.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#217

Post by suchow »

tantalor wrote: post #190
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

> doesn't work as well in chess: it works better That's not how "as well" works.

The misdirection is being used as a rhetorical device — you're supposed to feel a brief confusion when you get to the colon; it's then quickly resolved.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#218

Post by sillysaurus3 »

rditooait wrote: post #215
sillysaurus3 wrote: post #214

Earlier quoted context omitted.

Uhh... These games are actually broken. From the second link: https://imgur.com/a/P5tG6 See for yourself: https://lichess.org/Zqwn4Gzk#87 https://lichess.org/Zqwn4Gzk#88 EDIT: Nope, I'm just a noob.

https://en.wikipedia.org/wiki/En_passant

Ah, thanks.

I'm delighted. Chess seemed so simple. I had no idea there was a special pawn capture.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#219

Post by [deleted] »

sillysaurus3 wrote: post #214
soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

Uhh... These games are actually broken. From the second link: https://imgur.com/a/P5tG6 See for yourself: https://lichess.org/Zqwn4Gzk#87 https://lichess.org/Zqwn4Gzk#88 EDIT: Nope, I'm just a noob.

[deleted]

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#220

Post by forgot-my-pw »

sireat wrote: post #168
soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

Time and again Alpha shows it is much better at eval than Stockfish. Alpha play feels "human" at least to this FM. This is fantastic news! It is what I would imagine a good correspondence GM would play like with engine assistance. I already commented on Game 1 where Stockfish played extremely aggressively with 13. Ncxe5 ??! and 31. Qxc7 ?! Game 3 is a positional masterpiece. Alpha is willing to play pawns + exchange…

Thanks for the analysis.

There's a project currently that emulates AlphaGo Zero using distributed computing / crowdsourcing: https://github.com/gcp/leela-zero . You can run it on the browser too and it will submit the games after: https://ntt123.github.io/leela-zero/

Hope such a project will be available soon for the chess variant.

Or maybe Deepmind will release this as a SaaS product?

Post reply on HN