Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

161–170 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#161

Post by gcp »

bawana wrote: post #154
gcp wrote: post #129

Earlier quoted context omitted.

There are too many details missing for the results to be reproducible. Does it even qualify as a tech demo if the result only exists in DeepMind's lab?

This raises an interesting concept. If you cannot reproduce an experiment because of lack of resources, can you believe it? Or is this the equivalent of 'photoshopping your results'? A similar problem exists in cosmology. Can you verify the multiverse model if you only have one universe to experiment in? As data storage requirements in RAM and TPU power requirements increase to run certain models/algorithms, machine…

At least in computer chess such experiments were typically demonstrated by winning the World Championship. (And sometimes they failed...cough Deep Blue cough)

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#163

Post by bluecalm »

Recursing wrote: post #100
Invictus0 wrote: post #4

I'm not sure it's really fair to compare Stockfish to AlphaZero; AlphaZero used 24h of 5000 TPUs in compute time, and still needed 4 TPUs in real play, while Stockfish ran on just 64 threads and 1GB RAM. Nonetheless, still an impressive achievement.

Only 1GB RAM? Really?

Yes, this is really strange. Hash table size is a major contributing factor for strength of chess programs. It looks like a very artificial limitation.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#164

Post by make3 »

Houshalter wrote: post #21
gwern wrote: post #10

This is an incredible demonstration that the AG Zero expert iteration method is a general method. If you go back to the discussions of AG Zero lo a month ago, there was a lot of skepticism that NNs would ever challenge Stockfish et al - they are just too good, too close to perfection, and chess not well suited for MCTS and NNs. Well, it turns out that AG Zero doesn't work as well in chess: it works better as it only…

They aren't the first to apply NNs to chess though. What are they doing differently? And does anyone else smell smoke?

they are a huge company (Google) with access to top top top talent (experts) and infinite hardware resources. I don't know why it would be surprising if they acheived performance that hadn't been acheived before

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#165

Post by Aissen »

317070 wrote: post #134
Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

How do you replicate CERN experiments? The LHC? Hubble? LIGO? LISA? At least this paper is reproducible by people who have the compute, and many universities have super computers. Even at home, you can verify the results by replaying the games against stockfish. You might not be able to replicate the setup at home, but that does not mean it is not science.

Comparing projects done in the open with multiple different universities, on public funds, with something done behind closed doors with only personnel from a commercial entity is pretty far-fetched.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#166

Post by Aissen »

dandermotj wrote: post #140
Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

This is definitely a scientific paper. Pretty much no scientific paper comes with source code and the majority of scientific papers are not reproducible without an entire university department of resources anyway.

> Pretty much no scientific paper comes with source code

Are we blindly accepting this as science now ?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#168

Post by sireat »

soveran wrote: post #121

The ten sample games: Sample game 1 https://lichess.org/VMe0gfa2 Sample game 2 https://lichess.org/Zqwn4Gzk Sample game 3 https://lichess.org/G2fPHci8 Sample game 4 https://lichess.org/LLt8wyYp Sample game 5 https://lichess.org/3r6CXx3H Sample game 6 https://lichess.org/sbdyUYS4 Sample game 7 https://lichess.org/88vsAftE Sample game 8 https://lichess.org/1uvCwaeB Sample game 9 https://lichess.org/743quCXj Sample game…

Time and again Alpha shows it is much better at eval than Stockfish.

Alpha play feels "human" at least to this FM. This is fantastic news! It is what I would imagine a good correspondence GM would play like with engine assistance.

I already commented on Game 1 where Stockfish played extremely aggressively with 13. Ncxe5 ??! and 31. Qxc7 ?!

Game 3 is a positional masterpiece. Alpha is willing to play pawns + exchange down when it correctly evaluates that Black queen and rooks will be tied down.

This kind of long term thinking is beyond what regular engines perform.

Game 10 is also an impressive showing by Alpha. Alpha is willing to play down a piece and a pawn for 15 (30 ply) moves in a middle game beyond the reach of Stockfish's raw calculations.

If one could only get access to Alpha evals :) When do mere mortals get access to TPUs on Google Compute Engine?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#169

Post by gcp »

dandermotj wrote: post #140
Aissen wrote: post #106

Serious question: how does one evaluate the results reproducibility of this paper ? Maybe I'm missing some things but: - Are 1st gen TPUs even accessible ? You have to fill out a form to learn more about those second generation TPUs: https://cloud.google.com/tpu/ - I can't find the source code This does not look like a scientific paper, but a ( very impressive) tech demo.

This is definitely a scientific paper. Pretty much no scientific paper comes with source code and the majority of scientific papers are not reproducible without an entire university department of resources anyway.

...and this attitude causes quite a bit of them to not be reproducible even when people try with the same resources.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#170

Post by bhouston »

thom wrote: post #97
thom wrote: post #59

Earlier quoted context omitted.

The issue is you can’t evaluate positions that fast in AlphaZero (currently).

It would be interesting to see if there were some way to extract a couple of new heuristics from AlphaZero that could be implemented fast enough to incorporate in Stockfish's evaluator though. I suppose this is the age old problem of black-box models: _why_ does it think this?

I think that it is almost always possible to extract optimized models from nn and implement them faster. I wonder if this can be generalized. Nn to optimized fix algo for Max speed?

This has to already exist as it is very obvious.

Post reply on HN