Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

271–280 of 282 posts

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#271

Post by nojvek »

allenz wrote: post #248
backpropaganda wrote: post #137

Earlier quoted context omitted.

Next up: Google's Deepmind AI learns to perform arithmetic tabula rasa . More seriously, it seems Deepmind and the AI community in general is having a Streetlight effect problem, i.e. looking for AI in what works now, rather than coming to terms with the hard challenges. This explains why there are so many papers on GANs. People are just doubling down on what works (where the streetlight is), rather than acknowledgin…

I agree that the average Joe will misinterpret the significance of AlphaGo, to Google's benefit. But most people in the research community already know how amazing it would be to make an affordable household robot or a search-and-rescue robot or a self-driving car. Many labs (including mine) are working on it. The streetlight adds a small bias, but the bigger problem is that we have no idea how to build human-level A…

The biggest problem in robotics is vision. How do you translate pixels to a 3D scene graph with objects attributes and correlate with prior knowledge.

Do that in real time, on device without using a crazy amoubt of power because of batteries.

CNNs and faster GPUs are the biggest breakthrough in that regards but it's still a long way to go before we get to human level visual cortex.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#272

Post by nojvek »

forgot-my-pw wrote: post #220
sireat wrote: post #168

Earlier quoted context omitted.

Time and again Alpha shows it is much better at eval than Stockfish. Alpha play feels "human" at least to this FM. This is fantastic news! It is what I would imagine a good correspondence GM would play like with engine assistance. I already commented on Game 1 where Stockfish played extremely aggressively with 13. Ncxe5 ??! and 31. Qxc7 ?! Game 3 is a positional masterpiece. Alpha is willing to play pawns + exchange…

Thanks for the analysis. There's a project currently that emulates AlphaGo Zero using distributed computing / crowdsourcing: https://github.com/gcp/leela-zero . You can run it on the browser too and it will submit the games after: https://ntt123.github.io/leela-zero/ Hope such a project will be available soon for the chess variant. Or maybe Deepmind will release this as a SaaS product?

Deepmind should release the TF compatible model with weights. And then it's just a matter of shrinking the model enough to run on desktop hardware.

But I don't know whether they'll do it. I hope they follow suit like other researchers who have github repos with code and models besides their papers. Really accelerates research.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#273

Post by orasis »

wskish wrote: post #259

The more interesting metric going forward is performance at a given power budget (not unlike with motorsports). The TPUs are consuming sooo much power here! Most interesting real-world problems are power-limited, including in nature (e.g. metabolic limits).

When a lot of money is on the line you can use a lot of resources.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#274

Post by namarie »

narrator wrote: post #113

So when are they going to apply this to Atari Games or well anything? The next step is they have one AI figure out the rules by making a GAN that imitates player behavior and the other AI be Alpha Go which tweaks the GAN inputs to generate different moves to win. Voila...Almost General Purpose AI that can learn to play any game.

Didn't they start it all by playing atari games?

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#275

Post by bo1024 »

cdelsolar wrote: post #211

I wanted to contact the authors directly but can't seem to find contact info at the moment, with a question. I hope some of you might know enough to answer it. I'm interested in applying this method, or a similar neural-network / tabula rasa based method to the game of Scrabble. I read the original AlphaGo Zero paper and they mentioned that this method works best for games of perfect information. The standard Scrabbl…

Step 1: millions (?) of dollars of hardware.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#276

Post by jacksmith21006 »

modeless wrote: post #223
Retric wrote: post #198

Earlier quoted context omitted.

Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.

Several phones already have neural net acceleration hardware in them today, including the latest iPhones.

Same with the Pixel 2. But the Pixel 2 appears to be a bit more powerful than the iPhone neural chip. The PVC is able to do 3 TOPS but we really need instructions supported and word size to truly compare.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#277

Post by allenz »

nojvek wrote: post #271
allenz wrote: post #248

Earlier quoted context omitted.

I agree that the average Joe will misinterpret the significance of AlphaGo, to Google's benefit. But most people in the research community already know how amazing it would be to make an affordable household robot or a search-and-rescue robot or a self-driving car. Many labs (including mine) are working on it. The streetlight adds a small bias, but the bigger problem is that we have no idea how to build human-level A…

The biggest problem in robotics is vision. How do you translate pixels to a 3D scene graph with objects attributes and correlate with prior knowledge. Do that in real time, on device without using a crazy amoubt of power because of batteries. CNNs and faster GPUs are the biggest breakthrough in that regards but it's still a long way to go before we get to human level visual cortex.

Vision is part of the puzzle--a large part in the case of self-driving cars. But blind people are way better than computers at everyday tasks, so I don't think that it's the Big Problem.

Translating to 3D is low-level and relatively easy. That's not the reason why we don't have household robots/self-driving cars.

Framing vision as "object attributes" and "correlate to prior knowledge" might be a good approach for current research. But humans do more--we understand what we look at. We form concepts and models of the world that allow us to adapt to very novel situations.

The main reason why we haven't solved vision, language, playing chess like a human, etc is that NNs are a poor approximation of human concepts. I agree that we probably need more compute and better compute.

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#278

Post by apetresc »

kaffeemitsahne wrote: post #115
pacaro wrote: post #38

Earlier quoted context omitted.

Shogi is a fun game, it always feels a little sad that it doesn't get more exposure outside of Japan (and my understanding is that, by and large, in Japan it is considered an "old persons" game) Because captured pieces change sides, there is less of an "endgame" scenario, and as a beginner (like me) it is very easy to put too many captured pieces back into play, which makes it hard to defend everything and essentiall…

I've been interested in learning both shogi and xiangqi for a while. If anyone knows a nice engine with graphical frontend for either game, I'd love to know. Wasn't able to find much the last time I looked.

The best place to play Shogi online against others at http://81dojo.com/

Re: Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

#280

Post by Chickenosaurus »

skybrian wrote: post #227
varelse wrote: post #206

Earlier quoted context omitted.

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network…

I wonder how much the hardware would cost to rent for a researcher not working at Google?

Google provides 1000 TPUs free of charge to researchers https://www.tensorflow.org/tfrc/
Post reply on HN