Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

141–150 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#141

Post by Houshalter »

andbberger wrote: post #98
aeleos wrote: post #88

Earlier quoted context omitted.

That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence. EDIT: changed a set of neurons to neural network per andbbergers comments

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

Re: AlphaGo Zero: Learning from scratch

#142

Post by RoboTeddy »

shmageggy wrote: post #71

Looks like the performance improvement comes from two key ingredients: 1) Using Residual networks instead of normal convolutional layers 2) Using a smarter policy training loss that uses the full information from a MCTS at each move. In the previous version, I believe they just ran the policy network to the end of the game and used a very weak {0, 1} reinforcement signal over all of the moves played. Here, it looks l…

Also (3) training a dual policy & value network that can benefit from a single shared representation of the game

Re: AlphaGo Zero: Learning from scratch

#143

Post by namelost »

romaniv wrote: post #113
Scaevolus wrote: post #46

Earlier quoted context omitted.

They have a new reinforcement learning algorithm that should be generically applicable to anything where a long sequence of moves results in a specifically gradable outcome. > The neural network in AlphaGo Zero is trained from games of selfplay by a novel reinforcement learning algorithm. In each position s, an MCTS search is executed, guided by the neural network fθ. The MCTS search outputs probabilities π of playin…

> They have a new reinforcement learning algorithm that should be generically applicable to anything where a long sequence of moves results in a specifically gradable outcome. Statements like these always make me wonder why certain obvious things weren't tried. If it's so generic, why wasn't it tried on Chess? Or was it tried, failed to impress and thus didn't make it into the press release? This is a big problem wit…

The conventional wisdom for Chess engines is that aggressive pruning doesn't work well. Chess is much more tactical than Go, selective algorithms tend to lead to some crucial tactic being missed, and the greater the search depth, the more likely that is.

Modern Chess engines are designed to brute-force the search tree as efficiently as possible. I will go out on a limb here and say they would wipe the floor with AlphaGo, because AlphaGo's hardware would be more of a liability than an asset against a CPU.

See also: https://chessprogramming.wikispaces.com/Type+A+Strategy https://chessprogramming.wikispaces.com/Type+B+Strategy

Re: AlphaGo Zero: Learning from scratch

#144

Post by goatlover »

red75prime wrote: post #81
Arcsech wrote: post #20

Earlier quoted context omitted.

While this is promising, there's a long way to go between this and the other things you mentioned. Go is very well-defined, has an unequivocal objective scoring system that can be run very quickly, and can be simulated in such a way that the system can go through many, many iterations very quickly. There's no way to train an AI like this for, say, health: We cannot simulate the human body to the level of detail that'…

> most of the problems required for a general AI aren't well-defined. Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? > Producing a definitive, objective score for a paper clip collection is very difficult if not impossible. Erm, producing of objective comparison of relative values of Go board positions is still not possible.

> Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem?

You mean like go over and feed the neighbor's cat while they're on vacation?

How about instead, being able to clean any arbitrary building?

Go isn't remotely similar to the real world. It's a board game. A challenging one, sure, and AlphaGo is quite a feat, but it's not exactly translatable to open ended tasks with variable environments and ill-specified rules (maybe the neighbor expects you to know to water the plants and feed the goldfish as well).

Re: AlphaGo Zero: Learning from scratch

#145

Post by goatlover »

geofft wrote: post #28
ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

... yeah, before reading that link my position was "Wow, that's super neat, but Go is a pretty well-defined game," and after reading it I remembered that my position maybe a year or two ago was "Chess is a well-defined game that's beatable by AI techniques but Go is acknowledged to be much harder and require actual intelligence to play and won't be solved for a long while" and now I'm worried. Thanks for posting that…

Go is still a well defined game within a limited space that doesn't change, and rules that don't change. It's just harder than Chess, but that doesn't make it similar to tons of real world tasks humans are better at.

Re: AlphaGo Zero: Learning from scratch

#146

Post by lozenge »

Aron wrote: post #119

One idea occurs to me is to now evolve the Go game itself in a direction that adds more challenges for an AI to solve, and then solve those problems. How about being able to handle different and randomized board shapes? How about being allowed to say one move the opponent cannot take when you play a piece? It would be interesting to keep track of what variations the algorithm handles well automatically, and which it…

Arimaa was a chess inspired game intended to be difficult for computers. It "fell" in 2015.

Re: AlphaGo Zero: Learning from scratch

#148

Post by goatlover »

narvind wrote: post #16

How I wish Marvin Minsky would have stayed alive for one more year and seen this. He would have been so happy!

And he would have still said that deep learning lacks any sort of common sense understanding that's necessary to get close to human level intelligence.

Re: AlphaGo Zero: Learning from scratch

#149

Post by blt »

fragsworth wrote: post #106
etiam wrote: post #102

Earlier quoted context omitted.

Please don't refer to them as black boxes. The internals are fully observable.

It's a conceptual black box. There's no way for us to understand what each individual neuron is doing.

The tools we have developed so far are limited, but that's different from "there's no way". Many academics are working hard right now to better understand deep neural networks.

Re: AlphaGo Zero: Learning from scratch

#150

Post by ilaksh »

ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

The smoke is when things like the same simulated robot that learned to run around like a mentally challenged person also learns to simulate throwing and can read very basic language.

It will seem quite stupid and inept at first. So people will dismiss it. But when they have a system with general inputs and outputs that can acquire multiple different skills, that will be an AGI, and we can grow it's skills and knowledge passed human level.

Post reply on HN