Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

171–180 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#171

Post by Jach »

conistonwater wrote: post #153
Jach wrote: post #127

Earlier quoted context omitted.

I don't remember where I read it but in some earlier versions of AlphaGo they tried a komi of 6.5 and black ended up winning more often. That indicates the correct komi value is 7, but since Go doesn't have ties, you have to pick which side you want to favor to break the tie. (White seems reasonable.)

That doesn't sound right because in Chinese rules, which is what AG uses, komi only changes in steps of two. Are you sure about 6.5? Could it be 5.5?

Well in Japanese rules the komi is 6.5 so that's the alternative that tends to come up. Some quick searching I found a transcript from one of the games where DeepMind said 7.5 slightly favors white but they didn't say anything about 6.5 or 5.5, while a random comment from r/baduk claims that pro game analysis shows 6.5 slightly favors black and 7.5 slightly favors white.

Re: AlphaGo Zero: Learning from scratch

#172

Post by asah »

ohhhwell wrote: post #95
ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

Thanks for that link, well worth the read. This is an interesting question to ask in these "how far away is AGI" discussions: I was once at a conference where there was a panel full of famous AI luminaries, and most of the luminaries were nodding and agreeing with each other that of course AGI was very far off, except for two famous AI luminaries who stayed quiet and let others take the microphone. I got up in Q&A an…

IBM Watson on Winograd schemas? It beat jeopardy... ?

Re: AlphaGo Zero: Learning from scratch

#173

Post by indescions_2017 »

What's fascinating (and admittedly somewhat worrying) about Self-play is that an agent can accidentally become adept at tasks other than intended via transfer learning. The "wrestling spiders" in OpenAI's demo quickly mastered the art of Sumo Wrestling. And whatever skills they learned in resisting an opposing force to stay standing on a platform, were immediately applicable to myriad different domains. In this case, being subject to hurricane force winds, and not as any normal spider may, be hurled into the sky!

It's more difficult to see how Go playing skills can translate to other domains. But for tasks in robotics, cybersecurity or fintech the power of self-play trained transfer learning becomes more apparent.

Re: AlphaGo Zero: Learning from scratch

#174

Post by gwern »

mtremsal wrote: post #129
shmageggy wrote: post #71

Looks like the performance improvement comes from two key ingredients: 1) Using Residual networks instead of normal convolutional layers 2) Using a smarter policy training loss that uses the full information from a MCTS at each move. In the previous version, I believe they just ran the policy network to the end of the game and used a very weak {0, 1} reinforcement signal over all of the moves played. Here, it looks l…

How is it different to apply the loss on each actual move at the end of the game VS on each rollout (which is itself a tiny game)? Does it help reinforce learning towards the end game as shorter rollouts are needed? Is the more accurate information then propagated to earlier moves as well?

I think the difference is that under 1/0 policy gradient loss, it gets feedback only on the actual chosen move. Under MCTS-rollouts-each-move, it gets feedback on every move on the board whether its value estimate was slightly too high or low plus the ultimate outcome of the 1 move it did make.

Re: AlphaGo Zero: Learning from scratch

#175

Post by andbberger »

Houshalter wrote: post #141
andbberger wrote: post #98

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...

Re: AlphaGo Zero: Learning from scratch

#176

Post by goatlover »

tim333 wrote: post #166

Slightly scary how it went from zero to superhuman play in three days. I wonder if general AI will go that way one day.

General AI relative to an individual human, or billions of humans? The sum total of human beings, or organizations of humans is superhuman relative to an individual. We've had superhuman organizations for millennia. I'm not sure how much general AI will be different, other than the large scale automation of jobs which would happen.

As Rodney Brooks pointed out, all technology happens within a context, not a vacuum. A general AI will come to exist in a world with a lot of other superhuman capabilities already in existence.

Re: AlphaGo Zero: Learning from scratch

#178

Post by xoroshiro »

espadrine wrote: post #134
shmageggy wrote: post #58

Earlier quoted context omitted.

It still uses MCTS as its search algorithm. It no longer uses random rollouts as part of the evaluation, though. (Previously it was rollouts/2 + value_network/2)

Random rollouts are what the MC in MCTS stands for. Without that, it is simply a tree search. Excerpt from the paper: > [AlphaGo Zero] uses a simpler tree search that relies upon this single neural network to evaluate positions and sample moves, without performing any Monte-Carlo rollouts.

Does this mean it learns what to search? I wonder why they thought it was a good idea. I thought the whole point of MC was that pruning algorithms like the ones in chess wouldn't work for a larger search space.

Re: AlphaGo Zero: Learning from scratch

#180

Post by seanwilson »

auggierose wrote: post #179
seanwilson wrote: post #177

How do they evaluate that AlphaGo Zero is better than the previous AlphaGos? By playing them against each other? Or playing AlphaGo Zero against humans?

The same way they make AlphaGo Zero better, by playing against other AlphaGos.

But what if the other AlphaGos are blind to some tactic that humans could take advantage of? Not saying this is likely given how the Lee Sedol match and others went but I'm curious how they come up with the rankings.
Post reply on HN