Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

201–210 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#201

Post by sobellian »

tyler_larson wrote: post #197
sobellian wrote: post #193

The catch is that this isn't quite zero human knowledge, since the tree search algorithm is a human discovery, and not one that came easily to humans. It also massively cuts down on the search space for an appropriate policy function. That means that this setup isn't necessarily general. How applicable is MCTS to games with asymmetric information, a la Starcraft? What about games that can't quite be modeled with an a…

That's not quite what they're talking about WRT zero human knowledge. The problem is that there's no intrinsic scoring system for Go, nothing specific to maximize, so it's difficult to tell a computer whether a given outcome is "good" or "bad". So early versions of AlphaGo used a collection of human-played Go games to get an idea of what constitutes "good" and what is "bad", so it can then train its model to predict…

I understand that the paper means that they didn't train it on expert input. The significance of the research is that this is a more general way to construct a game AI. The question I am posing is how far we have to go on that front.

Re: AlphaGo Zero: Learning from scratch

#202

Post by roenxi »

goatlover wrote: post #144
red75prime wrote: post #81

Earlier quoted context omitted.

> most of the problems required for a general AI aren't well-defined. Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? > Producing a definitive, objective score for a paper clip collection is very difficult if not impossible. Erm, producing of objective comparison of relative values of Go board positions is still not possible.

> Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? You mean like go over and feed the neighbor's cat while they're on vacation? How about instead, being able to clean any arbitrary building? Go isn't remotely similar to the real world. It's a board game. A challenging one, sure, and AlphaGo is quite a feat, but it's not exactly translatable to open ended tas…

At this point, there is no evidence that the limiting factor in these cases is AI/software.

The limiting factor with the neighbors cat is the robotics of having a robust body and arm attachment. We know that the scope of current AI can:

1) Identify a request to feed a cat

2) Identify the cat, cat food and cat's bowl from camera data

3) Navigate an open space like a house

Being able to clean an arbitrary building is also more the challenge of building the robot than the AI identifying garbage on a floor or how to sweep something.

It is not clear there are hard theoretical limits on an AI any more. There are economic limits based on the cost of a programmer's attention. There are lots of hardware limits (including processor power).

Re: AlphaGo Zero: Learning from scratch

#203

Post by hasenj »

roenxi wrote: post #138
jasonwatkinspdx wrote: post #100

Earlier quoted context omitted.

> I wonder if this system produced more new styles of play. One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

The Go community learned to understand that the margin of victory is meaningless along time ago. The most famous game of Honinbo Dosaku, a famous Go player from the late 1600s, is arguably a game where he gave a handicap to his opponent and lost by one point. Lee Chang-Ho, who was the reigning champion in the late 90s, had a style that consistently tried to win by small margins. AlphaGo now appears to be better than…

I think if AlphaGo foresaw it was losing by one point, it would start playing reckless moves, as it did against Lee-Sedol in the only match it lost against him.

Re: AlphaGo Zero: Learning from scratch

#204

Post by geofft »

goatlover wrote: post #145
geofft wrote: post #28

Earlier quoted context omitted.

... yeah, before reading that link my position was "Wow, that's super neat, but Go is a pretty well-defined game," and after reading it I remembered that my position maybe a year or two ago was "Chess is a well-defined game that's beatable by AI techniques but Go is acknowledged to be much harder and require actual intelligence to play and won't be solved for a long while" and now I'm worried. Thanks for posting that…

Go is still a well defined game within a limited space that doesn't change, and rules that don't change. It's just harder than Chess, but that doesn't make it similar to tons of real world tasks humans are better at.

That's probably true, but that's very much not what people were saying about Go a couple years ago. There were a lot of people talking about how there isn't a straightforward evaluation function of the quality of a given state of the board, how things need to be planned in advance, how there's much more combinatorial explosion than in chess, etc., to the point where it's a qualitatively different game.

For me, as someone who accepted and believed these claims about Go being qualitatively different, realizing that no, it's not qualitatively different (or that maybe it is, but not in a way that impedes state-of-the-art AI research) is increasing my skepticism in other claims that board games in general are qualitatively different from other tasks that AIs might get good at.

(If you didn't buy into these claims, then I commend you on your reasoning skills, carry on.)

Re: AlphaGo Zero: Learning from scratch

#205

Post by sjg007 »

greysphere wrote: post #115
greysphere wrote: post #45

Earlier quoted context omitted.

TD-gammon is a well known version of this technique (with 2 ply lookahead, vs a 1600 deep mcts) https://en.m.wikipedia.org/wiki/TD-Gammon Temporal difference learning was previously consider weak at 'tactical' games, ie ones with gamestates that require long chains of precise moves to improve position (like many checkmate scenarios in chess) . For anyone more familiar with this technique, is it clear how the mcts/che…

After digging a bit deeper into the paper, it seems a key part of the new scheme is the NN is trained to help guide a deep/sparse tree search (as opposed to TD-gammons fully exhaustive 2-ply search). It's somewhat surprising to me that the simple win/loss is a strong enough signal to train this very 'intermediate step' in the algorithm - a spectacular result! It begs the question what other heuristic based algorithms…

It's estimating the probability of winning from the position based on what it has already seen. So basically it's a giant conditional probability distribution. Is it mistaken to interpret this as a bayesian network?

Re: AlphaGo Zero: Learning from scratch

#206

Post by tyler_larson »

sobellian wrote: post #201
tyler_larson wrote: post #197

Earlier quoted context omitted.

That's not quite what they're talking about WRT zero human knowledge. The problem is that there's no intrinsic scoring system for Go, nothing specific to maximize, so it's difficult to tell a computer whether a given outcome is "good" or "bad". So early versions of AlphaGo used a collection of human-played Go games to get an idea of what constitutes "good" and what is "bad", so it can then train its model to predict…

I understand that the paper means that they didn't train it on expert input. The significance of the research is that this is a more general way to construct a game AI. The question I am posing is how far we have to go on that front.

So... like this? https://www.theverge.com/2017/8/9/16117850/deepmind-blizzard...

Re: AlphaGo Zero: Learning from scratch

#207

Post by sobellian »

tyler_larson wrote: post #206
sobellian wrote: post #201

Earlier quoted context omitted.

I understand that the paper means that they didn't train it on expert input. The significance of the research is that this is a more general way to construct a game AI. The question I am posing is how far we have to go on that front.

So... like this? https://www.theverge.com/2017/8/9/16117850/deepmind-blizzard...

Yes. It'll be interesting to see if their starcraft project uses the same algorithms or not. Note that the link merely describes software that could be used for feature engineering. It doesn't describe what NN architecture or tree search algorithms deep mind is using.

Re: AlphaGo Zero: Learning from scratch

#208

Post by [deleted] »

geofft wrote: post #204
goatlover wrote: post #145

Earlier quoted context omitted.

Go is still a well defined game within a limited space that doesn't change, and rules that don't change. It's just harder than Chess, but that doesn't make it similar to tons of real world tasks humans are better at.

That's probably true, but that's very much not what people were saying about Go a couple years ago. There were a lot of people talking about how there isn't a straightforward evaluation function of the quality of a given state of the board, how things need to be planned in advance, how there's much more combinatorial explosion than in chess, etc., to the point where it's a qualitatively different game. For me, as som…

[deleted]

Re: AlphaGo Zero: Learning from scratch

#209

Post by red75prime »

andbberger wrote: post #175
Houshalter wrote: post #141

Earlier quoted context omitted.

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...

> Hinton is not a neuroscientist...

It's not like neuroscientists know that either.

Re: AlphaGo Zero: Learning from scratch

#210

Post by empath75 »

ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

You don't even need to produce an AGI for this kind of intelligence to be frightening.

At some point, a military is going to develop autonomous weapons that are vastly superior to human beings on the battle field, with no risk of losing human lives, and there is going to be a blitzkrieg sort of situation as the relative power of nations shifts dramatically.

If we have two such countries we could have massive drone and cyberwars being fought faster than people even can comprehend what's happening.

Right now most countries insist on maintaining human control over the machinery of death. But that will only last for as long as autonomous death machines don't dominate the battlefield.

It's a fun challenge right now to build a machine that can win in Starcraft, but it's really a hop skip and a jump from there to winning actual wars.

Post reply on HN