Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

251–260 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#251

Post by colllectorof »

gfodor wrote: post #227

I'm wondering if once one of these algorithms comes along that has been perfected if it is going to "burn in" the domain it was built for as the target of problem reductions, similar to 8086 assembly or the qwerty keyboard living on today despite them being ancient relics. For example, after this result it seems if you can reduce your problem domain onto Go (or a similarly structured game) you now have a way to creat…

In many domains mapping the problem to a tree search already gives you a superhuman solver or at least a passable solver. Problem mapping is what most of modern AI research is about. That's how the field was redefined in recent years. Just like Vladimir Vapnik says[1], it's becoming more engineering than science. (And sometimes more software alchemy than engineering.)

[1] https://www.youtube.com/watch?v=5mvfpSdWsOo "Brute Force and Intelligent Paradigms of Learning"

Re: AlphaGo Zero: Learning from scratch

#252

Post by infinity0 »

On the bright side, it means the several thousand years of humans playing Go, we were actually going "in the right direction" in terms of optimal strategy, despite not having reduced the game down into provably-optimal mathematical theorems.

Re: AlphaGo Zero: Learning from scratch

#254

Post by eutectic »

roenxi wrote: post #202
goatlover wrote: post #144

Earlier quoted context omitted.

> Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? You mean like go over and feed the neighbor's cat while they're on vacation? How about instead, being able to clean any arbitrary building? Go isn't remotely similar to the real world. It's a board game. A challenging one, sure, and AlphaGo is quite a feat, but it's not exactly translatable to open ended tas…

At this point, there is no evidence that the limiting factor in these cases is AI/software. The limiting factor with the neighbors cat is the robotics of having a robust body and arm attachment. We know that the scope of current AI can: 1) Identify a request to feed a cat 2) Identify the cat, cat food and cat's bowl from camera data 3) Navigate an open space like a house Being able to clean an arbitrary building is a…

I remember seeing a demonstration by a deep learning guy of a commercially available robot cleaning a house under remote control. You are seriously underestimating the difficulty of developing software to solve these problems in an integrated way.

Re: AlphaGo Zero: Learning from scratch

#255

Post by tom_wilde »

skinner_ wrote: post #80
Jyaif wrote: post #4

"It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features." This is amazing! The technology they came up with must be super generic.

Also, unsupervised. Also, no rollouts. They got rid of a lot of complexity. At this point it looks like a reasonable challenge to write a superhuman Go AI in 500 lines of unobfuscated python.

'rollouts' ELI5? I didnt pick this up from the paper..

thx :)

Re: AlphaGo Zero: Learning from scratch

#256

Post by auggierose »

seanwilson wrote: post #180
auggierose wrote: post #179

Earlier quoted context omitted.

The same way they make AlphaGo Zero better, by playing against other AlphaGos.

But what if the other AlphaGos are blind to some tactic that humans could take advantage of? Not saying this is likely given how the Lee Sedol match and others went but I'm curious how they come up with the rankings.

At the end of the paper they describe how the come up with ELO rankings, and that in order to avoid some bias due to self-play only they include the results of the AlphaGo's versus Fan, Sedol, etc.

Re: AlphaGo Zero: Learning from scratch

#257

Post by auggierose »

This is such an impressive result, and so general, I bet many people (including me) wish they knew exactly how to duplicate this result. It would be great if they created an online course that explained all algorithms in detail right up to the creation of AlphaGo Zero itself. The paper gives the impression that it shouldn't be too hard for them to create such a course.

Re: AlphaGo Zero: Learning from scratch

#258

Post by [deleted] »

btilly wrote: post #151
gizmo686 wrote: post #121

Earlier quoted context omitted.

You can only change komi by full point increments. There is a .5 to break ties, but a komi of 7.5 is identical to one of 7.4. From a theoretical standpoint, any non-integer komi should lead to one player winning 100% of the time. So even if the actual win ratio is 14:6 at komi=7.5 that might still be the best value.

If you had an estimate of the real difference, you could switch to breaking ties randomly. Black wins 60% of the ties, white wins 40%. There will be a ratio at which each side should win 50% of the time. I agree that with perfect play, it will be a 50% of a tie to each side. But it is still interesting to ask for a better estimate of practical play.

[deleted]

Re: AlphaGo Zero: Learning from scratch

#259

Post by Houshalter »

ollin wrote: post #212
Houshalter wrote: post #141

Earlier quoted context omitted.

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

>As far as we know the brain is just a "linear algebra blackbox"...Likely they use similar principles. I'm not an expert, but my impression is that this is not really a reasonable claim, unless you're only considering very small function-like subsystems of the brain (e.g. visual cortex). Neural nets (of the nonrecurrent sort) are strict feed-forward function approximators, whereas the brain appears to be a big mess o…

There are some parts of the brain we have no clue about. Episodic memory or our higher level ability to reason. But most of the brain is just low level pattern matching just like what NNs do.

The constraints you mention aren't deal breakers. We can make RNNs without maintaining a global state and fully unrolling the loop. See synthetic gradients for instance. NNs can do unsupervised learning as well, through things like autoencoders.

Re: AlphaGo Zero: Learning from scratch

#260

Post by shanusmagnus »

roenxi wrote: post #202
goatlover wrote: post #144

Earlier quoted context omitted.

> Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? You mean like go over and feed the neighbor's cat while they're on vacation? How about instead, being able to clean any arbitrary building? Go isn't remotely similar to the real world. It's a board game. A challenging one, sure, and AlphaGo is quite a feat, but it's not exactly translatable to open ended tas…

At this point, there is no evidence that the limiting factor in these cases is AI/software. The limiting factor with the neighbors cat is the robotics of having a robust body and arm attachment. We know that the scope of current AI can: 1) Identify a request to feed a cat 2) Identify the cat, cat food and cat's bowl from camera data 3) Navigate an open space like a house Being able to clean an arbitrary building is a…

In my opinion the deepest and most difficult aspect of this example is the notion of 'clean' which will be different across contexts. Abstractions of this kind are not even close to understood in the human semantic system, and in fact are still minimally researched. (I expect much of the progress on this to come from robotics, in fact.)
Post reply on HN