Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

251–260 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#251

Post by allenz »

allenz wrote: post #242
gwern wrote: post #200

Earlier quoted context omitted.

But better evaluation gives you asymptotic speedups. You can give Stockfish several times its computation (which is already a lot, I mean, 64 threads, come on) and it doesn't make good use of it since it just runs into the search wall. If you gave Stockfish the equivalent in CPU power (and I'm not sure this is a fair hypothetical since part of the appeal of NNs is that they have such efficient hardware implementation…

Absolutely. This required an exorbitant amount of compute, but DeepMind had to do novel, nontrivial research to make use of those resources.

Edit: DeepMind's victory over Stockfish didn't need novel research. Giraffe already demonstrated that the asymptotic speedup was possible; it just needed more compute.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#252

Post by allenz »

varelse wrote: post #206
allenz wrote: post #64

Earlier quoted context omitted.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

I don't think that the specific numbers are relevant for what deepnotderp and I were saying: that Giraffe already demonstrated the potential, and all that was missing was a boatload of compute.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#253

Post by red75prime »

allenz wrote: post #246
CydeWeys wrote: post #145

Earlier quoted context omitted.

> In that time I figure they used the equivalent of about 1000 cpu-years. Are you using some kind of conversion factor from TPUs to CPUs? If so, what is it? And is it valid to do that? You could convert the amount of time it took to render an hour's worth of gameplay from 1 GPU-hour to 50 CPU-days (or whatever), but is that really meaningful?

The conversion factor seems to be 1 TPU-hour ~ 500 CPU-hours in terms of flops. We can nitpick that number, but it won't change the conclusion that AlphaZero needs a boatload of compute.

It needs huge amount of computations to perform a task which previously required a boatload of domain experts' brainpower too.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#254

Post by sillysaurus3 »

XR0CSWV3h3kZWg wrote: post #247
sillysaurus3 wrote: post #204

Earlier quoted context omitted.

https://i.imgur.com/kwCyiHn.png That was a bad move for white to play. It's easy to win when your opponent throws the game. No human player would trade queens in that situation.

Are you thinking Qa5?

Yeah, or anything other than trading. You can see from the graph that it was all downhill from there; deservedly so.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#255

Post by aidenn0 »

maxander wrote: post #24
magoghm wrote: post #9

"We also analysed the relative performance of AlphaZero’s MCTS search compared to the state-of-the-art alpha-beta search engines used by Stockfish and Elmo. AlphaZero searches just 80 thousand positions per second in chess and 40 thousand in shogi, compared to 70 million for Stockfish and 35 million for Elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more…

Meanwhile a human player considers But unsettlingly few, nonetheless.

Strong human players consider a lot more than 1 position per second in Chess...

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#256

Post by joshuamorton »

allenz wrote: post #249
joshuamorton wrote: post #52

Earlier quoted context omitted.

But it looks like AlphaGo is searching fewer positions per second than Giraffe did. AlphaZero evaluates 80K positions per second, according to this paper, and the Giraffe paper says that Giraffe averaged 258570 evaluations per second when running STS. While we can't directly compare the computer power, this implies that AZ has learned a better representation.

It's unclear how much of the "better representation" was due to better algorithm vs. more compute/deeper NN.

Giraffe was trained until convergence. Maybe if there was more compute power then, a different model would have been used, but that's deep into the world of silly hypotheticals.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#257

Post by CydeWeys »

allenz wrote: post #246
CydeWeys wrote: post #145

Earlier quoted context omitted.

> In that time I figure they used the equivalent of about 1000 cpu-years. Are you using some kind of conversion factor from TPUs to CPUs? If so, what is it? And is it valid to do that? You could convert the amount of time it took to render an hour's worth of gameplay from 1 GPU-hour to 50 CPU-days (or whatever), but is that really meaningful?

The conversion factor seems to be 1 TPU-hour ~ 500 CPU-hours in terms of flops. We can nitpick that number, but it won't change the conclusion that AlphaZero needs a boatload of compute.

I don't see how this is relevant though. A GPU also provides graphics rendering performance equivalent to some boatload of CPU-hours, but who cares? GPUs exist and are used for the tasks they are good at. TPU hardware isn't theoretical; it does exist and it is being mass-produced.

Yes, it needs a boatload of very simple compute (8 bit operations), the kind that CPUs are not even close to ideal at providing economically.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#258

Post by CydeWeys »

Retric wrote: post #250
CydeWeys wrote: post #236

Earlier quoted context omitted.

Again, this entire article and discussion is about state-of-the-art chess. As in, literally working to "solve" the game and develop optimal strategy. I don't understand what relevance casual mobile chess games have. Computer chess is already very far beyond human capabilities, and it can't be pressed further just using mobile phone hardware (nor is that a reasonable restriction). It'd be like in a discussion about Sp…

That's not why this is relevant. Given equivalent hardware it's still a worse solution for chess . The value is you can get results of similar quality with vastly more compute power even without 1,000+ years of analysis. Otherwise the only takeaway is this failed to improve the state of the art.

"Equivalent hardware" is only relevant if we're talking about cost. When measured by that metric, the TPUs are indeed superior. Raw operations is an irrelevant metric given the existence of economic purpose-specific hardware that can perform a lot more of the operations required for matrix multiplication than for general computation. GPUs work exactly the same.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#259

Post by wskish »

The more interesting metric going forward is performance at a given power budget (not unlike with motorsports). The TPUs are consuming sooo much power here! Most interesting real-world problems are power-limited, including in nature (e.g. metabolic limits).

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#260

Post by defertoreptar »

luckyt wrote: post #28

It doesn't seem to like the Sicilian Defense (1.e4 c5), which is the most popular opening by human players. I wonder if this will change opening theory?

I thought it was interesting that it seems to like the English Opening. It's not popular, but Bobby Fischer played it in the world championship against Spassky.
Post reply on HN