Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

231–240 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#231
post #198

When things will start getting interesting is when we figure out how to get move simulation and search into the network itself, rather than programming that on the outside. As far as I know, no-one has even the faintest idea of how to do that. We have an existence proof that this should be possible. The networks are great at perception and snap-prediction. Anything a human can do in 200ms is fair game. And with cleve…

> We have an existence proof that this should be possible.

Not guaranteed. The human brain has diffusion signalling (i.e. neurotransmitters passing out of the synaptic cleft, into a neighbouring one, and activating a receptor on some other spacially-local axon as a result.) And one of those signalling molecules is thought to represent, in its intensity, a confidence-interval bias adjustment (i.e. a pruning bias factor for MCTS.) So the brain's MCTS-equivalent process may rely on some extra-graphical properties of the brain-as-embodied-meat-thing.

Re: AlphaGo Zero: Learning from scratch

#232
post #227

I'm wondering if once one of these algorithms comes along that has been perfected if it is going to "burn in" the domain it was built for as the target of problem reductions, similar to 8086 assembly or the qwerty keyboard living on today despite them being ancient relics. For example, after this result it seems if you can reduce your problem domain onto Go (or a similarly structured game) you now have a way to creat…

I think the only thing about go that enables this technique is "turn-based perfect information 0-sum game".

Re: AlphaGo Zero: Learning from scratch

#233

What's fascinating (and admittedly somewhat worrying) about Self-play is that an agent can accidentally become adept at tasks other than intended via transfer learning. The "wrestling spiders" in OpenAI's demo quickly mastered the art of Sumo Wrestling. And whatever skills they learned in resisting an opposing force to stay standing on a platform, were immediately applicable to myriad different domains. In this case,…

It is clear that these "self-play" scenarios depend on simulation - unless there is an appropriate stage for self play to take place on, there can be no play. The question is - how do we stand with simulation for robotics, self driving cars, etc.

My bet is that simulation is going to be the crowning jewel in the AI field, replacing static datasets and supervised learning with "dynamic datasets" and rewards. It would help with data sparsity as well (where can you find an image of a donkey riding an elephant for the new ImageNet? - but you can sim that or any possible combination).

Not to mention that humans are fallen head over heels with simulation as well - VR headsets and games in general. I see a great future for simulation with both AI and humans. It will be our common learning/playing/research sandbox.

Re: AlphaGo Zero: Learning from scratch

#234
post #54
post #48

Earlier quoted context omitted.

But I suppose they still do the searching/pruning with a separate piece of code (not a neural network).

Yes, but tree search + neural net is still pretty generic. It only assumes that you can enumerate branches.

You're right, but MC rollouts work better (better estimate) for some games than others.

Re: AlphaGo Zero: Learning from scratch

#235
post #232
post #227

I'm wondering if once one of these algorithms comes along that has been perfected if it is going to "burn in" the domain it was built for as the target of problem reductions, similar to 8086 assembly or the qwerty keyboard living on today despite them being ancient relics. For example, after this result it seems if you can reduce your problem domain onto Go (or a similarly structured game) you now have a way to creat…

I think the only thing about go that enables this technique is "turn-based perfect information 0-sum game".

Also it has fairly limited input. A real world problem may have much more possible inputs at any time step as opposed to placing just 1 stone

Re: AlphaGo Zero: Learning from scratch

#236
post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

You don't even need to produce an AGI for this kind of intelligence to be frightening. At some point, a military is going to develop autonomous weapons that are vastly superior to human beings on the battle field, with no risk of losing human lives, and there is going to be a blitzkrieg sort of situation as the relative power of nations shifts dramatically. If we have two such countries we could have massive drone an…

Nuclear ICBMs already push us past that boundary. The world can no longer afford to fight a war seriously.

Re: AlphaGo Zero: Learning from scratch

#237
post #204

Earlier quoted context omitted.

Go is still a well defined game within a limited space that doesn't change, and rules that don't change. It's just harder than Chess, but that doesn't make it similar to tons of real world tasks humans are better at.

That's probably true, but that's very much not what people were saying about Go a couple years ago. There were a lot of people talking about how there isn't a straightforward evaluation function of the quality of a given state of the board, how things need to be planned in advance, how there's much more combinatorial explosion than in chess, etc., to the point where it's a qualitatively different game. For me, as som…

About those claims- this is from Russel and Norvig, 3d ed. (from 2003, so a way back):

Go is a deterministic game, but the large branching factor makes it challeging. The key issues and early literature in computer Go are summarized by Boozy and Cazenave (2001) and Muller (2002). Up to 1997 there were no competent Go programs. Now the best programs play most of their moves at the master level; the only problem is that over the course of a game they usually make at least one serious blunder that allows a strong opponent to win. Whereas alpha—beta search reigns in most games, many recent Go programs have adopted Monte Carlo methods based on the UCT (upper confidence bounds on trees) scheme (Kocsis and Szepesvari, 2006). The strongest Go program as of 2009 is Golly and Silver's MoGo (Wang and Golly, 2007; Gelly and Silver, 2008). In August 2008, MoGo scored a surprising win against top professional Myungwan Kim, albeit with MoGo receiving a handicap of nine stones (about the equivalent of a queen handicap in chess). Kim estimated MOGO's strength at 2-3 dan, the low end of advanced amateur. For this match, MoGo was run on an 800-processor 15 terailop supercomputer (1000 limes Deep Blue). A few weeks later, MoGo, with only a five-stone handicap, won against a 6-dan professional. In the 9 x 9 form of Go, MoGo is at approximately the 1-dan professional level. Rapid advances are likely as experimentation continues with new forms of Monte Carlo search. The Computer Go Newsletter, published by the Computer Go Association, describes current developments.

There's no word about how Go is qualitatively different to other games, but maybe the referenced sources say something along those lines. Personally, I took a Masters course in AI two years ago, before AlphaGo and I remember one professor saying that the last holdout where humans can still beat computers in board games was GO, but I don't quite remember him saying anything about qualititative difference. Still, I can recall hearing about the idea that Go needs intuition or something like that, except I've no idea where I've heard that. I guess it might come from the popular press.

I guess this will sound a bit like the perenial excuse that "if it works, it's not AI" but my opinion about Go is that humans just weren't that good at it, after all. We may have thought that we have something special that makes us particularly good at Go, better than machines- but AlphaGo[Zero] has shown that, in the end, we just have no idea what it means to be really good at it (which, btw, is a damn good explanation of why it took us so long to make AI to beat us at it).

That, to my mind, is a much bigger and much more useful achievement than making a good AI game player. We can learn something from an insight into what we are capable of.

Re: AlphaGo Zero: Learning from scratch

#238
post #231
post #198

When things will start getting interesting is when we figure out how to get move simulation and search into the network itself, rather than programming that on the outside. As far as I know, no-one has even the faintest idea of how to do that. We have an existence proof that this should be possible. The networks are great at perception and snap-prediction. Anything a human can do in 200ms is fair game. And with cleve…

> We have an existence proof that this should be possible. Not guaranteed. The human brain has diffusion signalling (i.e. neurotransmitters passing out of the synaptic cleft, into a neighbouring one, and activating a receptor on some other spacially-local axon as a result.) And one of those signalling molecules is thought to represent, in its intensity, a confidence-interval bias adjustment (i.e. a pruning bias facto…

That will be a couple of additional terms in activation function. Or am I missing something?

Re: AlphaGo Zero: Learning from scratch

#239
post #191

Earlier quoted context omitted.

The conventional wisdom for Chess engines is that aggressive pruning doesn't work well. Chess is much more tactical than Go, selective algorithms tend to lead to some crucial tactic being missed, and the greater the search depth, the more likely that is. Modern Chess engines are designed to brute-force the search tree as efficiently as possible. I will go out on a limb here and say they would wipe the floor with Alph…

Until I see AlphaGo zero defeating StockFish 100-0 and with same algorithm defeating best Go AI and killing the Atari games including montezuma’s revenge, I call this hype bullshit. Give me your results on OpenAI gym in a variety of different styles of games including GTA and WoW. I will believe you if a generic unsupervised algorithm running on a single machine is absolutely destroying the best players. Until then .…

I don't know if you're being sarcastic or not. If not, I suggest you look at the cartoon at http://www.kurzweilai.net/robot-learns-self-awareness

Re: AlphaGo Zero: Learning from scratch

#240
post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

I spent an hour of my life that I'll never get back reading Yudkowski's overly-long article and I believe I can summarise it thusly: "We don't know how AGI will arise; we don't know when; we don't know why; we don't know anything at all about it and we won't know anything about it until it's too late to do anything anyway; We must act now!!" The question is- if we don't know anything about this unknowable threat, how…

So, in your view, starting MIRI, doing fundamental research into AI safety and advocating for it, is not trying to find the damn light?

You exemplify exactly the attitude he's trying to combat. "Oh, nobody knows anything, let's not care about consequences and do whatever."

Post reply on HN