Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

71–80 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#71

Post by grumpopotamus »

zwischenzug wrote: post #61

I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…

The Stockfish engine provided by lichess on the game you linked doesn't seem to mind those moves - it has most of them in the top few lines after a few seconds of thinking time.

Qe1 and Kh1 are fine if the plan is to prepare f4.

35. Nc4 stuck around at the #2 / #3 best move for as long as I ran that position.

Remember the Stockfish in the paper had 64 cores so you'd have to run your Stockfish for a while to get it to arrive at the same principle variation.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#72

Post by zwischenzug »

zwischenzug wrote: post #61

I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…

If I leave Stockfish to study for longer then Qe1 comes up in the analysis. Which makes me wonder whether SF gets weaker in some positions the more it's left to think.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#73

Post by zwischenzug »

grumpopotamus wrote: post #71
zwischenzug wrote: post #61

I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…

The Stockfish engine provided by lichess on the game you linked doesn't seem to mind those moves - it has most of them in the top few lines after a few seconds of thinking time. Qe1 and Kh1 are fine if the plan is to prepare f4. 35. Nc4 stuck around at the #2 / #3 best move for as long as I ran that position. Remember the Stockfish in the paper had 64 cores so you'd have to run your Stockfish for a while to get it to…

Yeah that's right. I think this might say more about the efficacy of chess engines over a certain point vs human analysis rather than the 'bullshit' I called.

I'd certainly fancy my chances against this AI more than Stockfish on a lower power.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#74

Post by visarga »

roenxi wrote: post #49
visarga wrote: post #11

Earlier quoted context omitted.

Shogi, chess and Go are "perfect information games", meaning you can see the whole game state. It's a whole different thing to be able to solve games where you don't see everything (based on uncertainty).

Is it really though? A big class of imperfect information games can be modeled by having a record of everything the agent has seen so far. Then it has exactly the same, if not more, information available than a human player in the same position. We know that with equal information AIs can make better decisions than humans (see also, AlphaGo :] ) so at that point the AI could reasonably be expected to achieve superhum…

Could work, but it hasn't been widely demonstrated yet. I really hope we can tackle such games/RL tasks.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#75

Post by dnsva »

luckyt wrote: post #28

It doesn't seem to like the Sicilian Defense (1.e4 c5), which is the most popular opening by human players. I wonder if this will change opening theory?

Combinations appear in Sicilian often. If you're playing black, that's a good thing depending on your rating.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#76

Post by sanxiyn »

Cybiote wrote: post #69
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

Let's break this down and consider things carefully. To informed researchers, what is most surprising here is not that the AlphaGo Zero algorithm beat stockfish but that MCTS managed to outperform Alpha-beta search. I'll venture a hypothesis as to why this was. Informed skepticism would have discounted MCTS against alpha-beta search but wouldn't have put much stock into the idea that Neural Networks couldn't learn be…

I agree that Searn is rather prophetic in retrospective.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#77

Post by mikekchar »

foobaw wrote: post #56
pacaro wrote: post #38

Earlier quoted context omitted.

Shogi is a fun game, it always feels a little sad that it doesn't get more exposure outside of Japan (and my understanding is that, by and large, in Japan it is considered an "old persons" game) Because captured pieces change sides, there is less of an "endgame" scenario, and as a beginner (like me) it is very easy to put too many captured pieces back into play, which makes it hard to defend everything and essentiall…

It briefly became popular in the otaku culture from an anime called Hunter X Hunter.

I'm curious to see if "San Gatsu no Lion" (the Lion of March) will spark interest. I highly recommend it to anyone interested in more slice-of-life/drama kinds of things. It's quite a beautiful anime/manga, even if the shogi isn't quite centre stage.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#78

Post by badminton1 »

gamegoblin wrote: post #43
badminton1 wrote: post #5

If you have seen the Stockfish project you will see many hardcoded weights in the configuration, found through experimentation. All these adjustments took probably years to achieve... and now Alpha Go Zero just self-learns everything and surpasses it. Would be good to see Deepmind's solution play Arimaa and Stratego, and see what kind of strategy it comes up with. Or weird variations of Go. Eventually this tech will…

I don't think this technique immediately applies to Stratego because it's not a perfect information game. I suspect it would exceed the state of the art in Arimaa, since Arimaa is specifically designed to have a high branching factor (17281 -- compared to 35 for chess), and this technique was designed to work well in high-branching factor games (since Go is a high-branching factor game, though much lower than Arimaa)…

In that regard then Stratego would share some aspects with Starcraft, another incomplete information game.

Deepmind is actively working in a StarCraft bot. It would be interesting to see if they can be put together a supraintelligent StarCraft bot and then translate those results to Stratego.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#79

Post by zwischenzug »

zwischenzug wrote: post #72
zwischenzug wrote: post #61

I smell a rat. The paper says: 'AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi' In the first game Stockfish's, 9. Qe1 is one of the strangest moves I've ever seen, which would never be considered by a human, let alone a superhuman. 11. Kh1 also makes little sense, but is not as bad. My Stockfish sees it as losing 0.2 pawns, which makes it highly suspect in such a positio…

If I leave Stockfish to study for longer then Qe1 comes up in the analysis. Which makes me wonder whether SF gets weaker in some positions the more it's left to think.

Now I'm really intrigued.

SF plays really odd moves when left to its own devices for a time. As does this AI. So maybe chess looks really weird with play significantly better than the best humans.

It's actually really disturbing.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#80

Post by thom »

zwischenzug wrote: post #79
zwischenzug wrote: post #72

Earlier quoted context omitted.

If I leave Stockfish to study for longer then Qe1 comes up in the analysis. Which makes me wonder whether SF gets weaker in some positions the more it's left to think.

Now I'm really intrigued. SF plays really odd moves when left to its own devices for a time. As does this AI. So maybe chess looks really weird with play significantly better than the best humans. It's actually really disturbing.

I think being able to play tactically perfect chess over 20 or so moves will often look weird to human strategic sensibilities. The computer sees every tiny exception to the patterns and heuristics you've incorporated into your gut feel about positions. In a way these moves are right just because they're right, and that's what's jarring - there's no _principle_ behind them that can be learned and generalised, which is something humans struggle with in all walks of life.
Post reply on HN