Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MuZero: Mastering Go, chess, shogi and Atari without rules

deepmind.com

Post reply on HN
Print view View original

71–79 of 79 posts

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#71

Post by mckirk »

martamorena284 wrote: post #61
philipkglass wrote: post #16

Earlier quoted context omitted.

A machine controlling a real car gathers feedback no faster than real time and cannot afford to learn the meanings of street signs from the consequences of ignoring them. A machine can learn the rules of Atari games from scratch by playing them orders of magnitude faster than real time and treating "death" as one signal among many. In order for a machine to learn driving the same way it learns Atari games, it seems t…

Really not true. We have all the tools available... Just model a city in Unreal Engine 4 (GTA 6 anyone?). Your sensors, like LIDAR could do ray-queries, cameras could render views, etc. It's not 100% real, but probably real enough to learn the basics. Photorealistic graphics should be sufficient for AI to learn real-world interaction. In the end our whole world might be just that, a simulation. So I see zero reason w…

The problem is that these simulations are far from perfect, and AIs are great at exploiting paths-of-least-resistance. So you'd end up with an AI that could drive through your virtual city flawlessly by overfitting on some cues that you wouldn't even notice, but are reliable enough because of the necessarily lower complexity of the simulation.

It wouldn't translate to the real world; unless maybe you add enough noise to the simulation to prevent the AI from using too simple cues, but at that point, it's questionable how much insight the AI could still distill from the simulation.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#72

Post by Jabbles »

syntheticmindai wrote: post #37
mark_l_watson wrote: post #36

I watched the Alpha Go vs. Lee Sedol games live. Big fan. That said, I think Deep Mind should go all in for solving practical real world problems.

They have used muzero to do video compression and saved 5% of bits. Source: david silver wired.co.uk interview

I couldn't find that source, please elaborate.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#73

Post by nightcracker »

thomasahle wrote: post #70
nightcracker wrote: post #47

Earlier quoted context omitted.

> while a weak enough player may consider illegal moves while planning in your head This isn't just weak players. E.g. strong chess players often consider moves as if blocking pawns weren't there. They might consider a bishop to be on a strong diagonal despite there being a blocking pawn because they can imagine moves that would happen if that pawn would disappear.

I suppose you are right. But MuZero won't be able to do this, since it's training forces it to consider legal moves in its planning.

No it doesn't. MuZero does its planning entirely in its own latent space (it may not even actually think of the game in terms of 'moves' but in whatever steps it considers relevant instead), only the output is filtered for legal moves.

It's no different than a monkey operating a chess computer that makes sure the monkey only performs legal moves. Your suggestion would be akin to suggesting that the chess computer would be affecting the monkey's mind so that it can only think in terms of legal chess moves.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#74

Post by nightcracker »

confuseshrink wrote: post #56
nightcracker wrote: post #48

Earlier quoted context omitted.

Eh, I'd say it's fun if you have a couple thousand TPUs lying around. If you're just messing around with 1 GPU and a desktop PC you should be happy to get Atari breakout to work.

The published hyperparameters are usually ridiculously conservative, for the simple games like breakout and pong you can usually converge in far fewer frames than in the papers.

I have tried reproducing the papers, with mixed success. I do not share your sentiment.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#75

Post by Shorel »

philipkglass wrote: post #16
bko wrote: post #9

Earlier quoted context omitted.

I've noticed all the top performing AI reinforcement algorithms i hear about know next to nothing about the initial rules. And not only do they perform as well as more supervised methods, but much better The one exception is self driving. I listened to the Lex Fridman interview with ceo of waymo recently and he made a case for the controlled environment (e.g. separate detection from decision making and planning) and…

A machine controlling a real car gathers feedback no faster than real time and cannot afford to learn the meanings of street signs from the consequences of ignoring them. A machine can learn the rules of Atari games from scratch by playing them orders of magnitude faster than real time and treating "death" as one signal among many. In order for a machine to learn driving the same way it learns Atari games, it seems t…

In that case, I would like to see what the AI can learn in Euro Truck Simulator 2.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#76

Post by Shorel »

kadoban wrote: post #10
deegles wrote: post #5

Is there a way to learn Go from scratch using these AIs? I wonder if it would pay off in the long run to be fully trained by one.

You can play against open source reimplementations of some of the ideas behind AlphaGo family AIs. LeelaZero was one of the early ones, KataGo is probably your best bet right now. Sai is also in the mix. All are _very_ strong. KataGo is ungodly strong, it beats pros. Learning Go is about more than just playing against strong players, but it could help. The biggest difficulty is that the strong AIs aren't actually tha…

> All are _very_ strong.

That's the problem. To learn, we need an AI that can be just a little bit stronger than humans, but at the same time we need an AI that makes natural moves, not an AI that makes great moves 90% of the time and clear blunders 10% of the time.

And playing go, the AI should be able to give handicap stones and play reasonable teaching moves.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#78

Post by mark_l_watson »

tobessebot wrote: post #49
mark_l_watson wrote: post #36

I watched the Alpha Go vs. Lee Sedol games live. Big fan. That said, I think Deep Mind should go all in for solving practical real world problems.

I mean they pretty much solved protein folding this year...

+1 Yes, that is impressive and practical.

Re: MuZero: Mastering Go, chess, shogi and Atari without rules

#79

Post by vermilingua »

willowwonder45 wrote: post #77
vermilingua wrote: post #41

DeepMind seems to be building the Wintermute to OpenAI’s Neuromancer. Where’s Turing?

Curious as to what is "Turing" in this context?

In Neuromancer (William Gibson's 1984 genre-defining cyberpunk novel), the Turing Police enforce laws prohibiting the creation of any superintelligent AI. I don't want to spoil anything, so we'll leave it there.
Post reply on HN