Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

131–140 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#131

Post by gcp »

gamegoblin wrote: post #94
asdfologist wrote: post #39

While this sounds impressive, I'll believe it when AlphaZero wins TCEC.

Back when AlphaGo was playing Lee Sedol I was thinking about a chess playing version in TCEC. The interesting thing is TCEC assumes a bit about the structure of the chess program. That is, the TCEC win-adjudication rule says that if both programs agree that one program is 6.5 pawns ahead for 8 turns in a row, they judge that program to be the winner. But programs like Alpha don't have an evaluation function that oper…

You can convert winning percentages to centipawns, so that's not a problem.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#132

Post by thanatropism »

elcapitan wrote: post #112

What would be a good starting point to learn about the AI behind that for a "normal" programmer? There seem to be so many resources now that it's hard to choose. Combination of hands-on plus theory would be good.

The keyword is "reinforcement learning".

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#133

Post by visarga »

backpropaganda wrote: post #47
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

While I'm glad to see you're excited about this, take note that this is still an approach which requires that an exact model is known, the state is fully visible, and the reward is perfectly define-able and known. Progress in this setup isn't necessarily correlated with the kind of AI for which we'd need a fire alarm.

It's easy for many to think that solving Go and chess means we can also solve household work like cleaning, cooking and washing dishes but it's actually harder.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#134

Post by 317070 »

Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

How do you replicate CERN experiments? The LHC? Hubble? LIGO? LISA? At least this paper is reproducible by people who have the compute, and many universities have super computers.

Even at home, you can verify the results by replaying the games against stockfish. You might not be able to replicate the setup at home, but that does not mean it is not science.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#135

Post by thanatropism »

gwern wrote: post #18
2bitencryption wrote: post #8

Earlier quoted context omitted.

> only to turn around and tell us that it is amazing that it can also win at Chess. What they're demoing here is a single, general formula for mastering multiple games. Start with empty AG0, then teach it chess from scratch until it is the strongest player on the planet. Go back to an empty slate, with the same exactly "untrained" AG0, and now teach it Go, to the same result. No fine-tuning for the domain of the game…

> question for someone who has time to read the paper: can you train it to master chess and go at the same time? or is it one or the other? I'm assuming the latter. I'm sure you could with a multi-headed NN. But what would be the point? There's very little transfer of knowledge between the games, especially once you get past the very most basics.

The point is that real problem domains are not neatly partitioned and labeled.

I don't know what kind of input the NN itself gets, but computer vision is enough to translate a photo of a chessboard to a usable symbolic representation. But it would be nice to already have a black box-ish computer program that figures out what's the game at hand and how to play it.

The next variation is have the adversary start playing a chess variant and have the machine recognize it (assuming honesty) and play it to significant skill. Then "real life Pong" where the size and aerodynamics of the ball are unknown to it. This is the gist of human intelligence: answering questions is significantly easier than figuring out what the question is.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#136

Post by vinchuco »

namelost wrote: post #53
computerphage wrote: post #42

Earlier quoted context omitted.

It beat the winner of TCEC-2016, Stockfish, with a record of 28-72-0. That's zero losses.

They didn't demonstrate that AlphaGo Zero can beat Stockfish in a fair contest: i.e. take the amount of money they spent on Stockfish's CPU and RAM, buy a commodity GPU for AlphaGo and then see.

https://news.ycombinator.com/item?id=2232163

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#137

Post by backpropaganda »

visarga wrote: post #133
backpropaganda wrote: post #47

Earlier quoted context omitted.

While I'm glad to see you're excited about this, take note that this is still an approach which requires that an exact model is known, the state is fully visible, and the reward is perfectly define-able and known. Progress in this setup isn't necessarily correlated with the kind of AI for which we'd need a fire alarm.

It's easy for many to think that solving Go and chess means we can also solve household work like cleaning, cooking and washing dishes but it's actually harder.

Next up: Google's Deepmind AI learns to perform arithmetic tabula rasa.

More seriously, it seems Deepmind and the AI community in general is having a Streetlight effect problem, i.e. looking for AI in what works now, rather than coming to terms with the hard challenges. This explains why there are so many papers on GANs. People are just doubling down on what works (where the streetlight is), rather than acknowledging that where we need to look for AI is dark. Since it's become such a cut-throat race to be the next one to say "we made a breakthrough!", it makes much more economic sense to solve simple problems and advertise them as huge challenges.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#138

Post by tom_wilde »

elcapitan wrote: post #112

What would be a good starting point to learn about the AI behind that for a "normal" programmer? There seem to be so many resources now that it's hard to choose. Combination of hands-on plus theory would be good.

Coursera - Andrew NG's course => Classic starting point, very thorough and digestable introduction to Neural Networks. I found he covered the 'how the heck do I use this?' rather well.. :)

From there, Coursera has a paid(?) DL course by Andrew NG or there's Fast.ai which looks good.

Good luck!

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#139

Post by elcapitan »

thanatropism wrote: post #132
elcapitan wrote: post #112

What would be a good starting point to learn about the AI behind that for a "normal" programmer? There seem to be so many resources now that it's hard to choose. Combination of hands-on plus theory would be good.

The keyword is "reinforcement learning".

I know the names of the general concepts, I was wondering if someone has concrete recommendations on where to start and which books/frameworks are sort of beginner-friendly.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#140

Post by dandermotj »

Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

This is definitely a scientific paper. Pretty much no scientific paper comes with source code and the majority of scientific papers are not reproducible without an entire university department of resources anyway.
Post reply on HN