Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

181–190 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#181

Post by andbberger »

aeleos wrote: post #130
andbberger wrote: post #98

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

I changed my comment to neural network since a set of neurons is somewhat wrong, but I don't really agree that there isn't much of a connection between this and biology. While there might not be much of a connection between how they currently work and how our brains work, the whole point of machine learning and neural networks is to improve computers performance on the things we are good at. And while originally it w…

I'm not saying there is nothing of value to be obtained by investigating connections between ML and the brain. That's how I got into ML in the first place, doing theoretical neuro research.

We absolutely should and do look to the brain for inspiration.

I'm taking issue with the rather ham-fisted series of papers that have come out in recent years aggressively pushing the agenda of connections between ML and neuro that just aren't there.

Are you sure that humans have done more net compute on Go than Deepmind just did? The Go game tree is _enormous_, humans are bias. We don't invent strategies from scratch, we use heuristics handed down to us from the pros (who in turn were handed down the heuristics from their mentors).

To me, it's not so interesting or surprising that the human initialized net performed worse. We just built the same biases and heuristics we have into that net.

Re: AlphaGo Zero: Learning from scratch

#182

Post by fspeech »

andbberger wrote: post #98
aeleos wrote: post #88

Earlier quoted context omitted.

That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence. EDIT: changed a set of neurons to neural network per andbbergers comments

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

Agreed. If we announce that A* search is superhuman in finding best routes, most technorati would't bat an eye. Technically it is probably accurate to say that the results here show that neural networks can find good heuristics for MCTS search through unsupervised training in the game of Go. According to DeepMind authors:

"These search probabilities usually select much stronger moves than the raw move probabilities of the neural network; MCTS may therefore be viewed as a powerful policy improvement operator. Self-play with search – using the improved MCTS-based policy to select each move, then using the game winner as a sample of the value – may be viewed as a powerful policy evaluation operator. The main idea of our reinforcement learning algorithm is to use these search operators repeatedly in a policy iteration procedure ..."

The fact that this reinforcement training is unsupervised from the very beginning is quite exciting and may lead to better heuristics for other kinds of combinatorial optimization problems.

Re: AlphaGo Zero: Learning from scratch

#184

Post by madez »

andbberger wrote: post #175
Houshalter wrote: post #141

Earlier quoted context omitted.

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...

If he changes his opinion, which I understand to be models of the brain in this case, and each iteration improves the model, then that is perfectly fine. It would be bad if someone did not change their view in case of inconsistent evidence.

Re: AlphaGo Zero: Learning from scratch

#185

Post by andbberger »

etiam wrote: post #102
andbberger wrote: post #98

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

Please don't refer to them as black boxes. The internals are fully observable.

Fully observable and we still have no idea what the hell it's doing.

Makes neuroscience seem kinda bleak doesn't it?

There has been a lot of great work lately building up a theory of how these things work, but it is very much still in the early stage. Jascha Sohl-Dickstein in particular has been doing some great work on this.

We don't even have answers to the most basic questions.

For instance (pedagogically), how the hell is it possible to train these things at all? They have ridiculously non-convex loss landscapes and we optimize in the dumbest conceivable way, first-order stochastic gradient descent. This should not work. But it does, all too often.

Not a great example because there are easy hand wavy arguments as to why it should work, but as far as proofs go...

The hand wavy argument goes as follows: - we're in like a 10000 dimensional space, for the stationary point we're at to be a true local minima that means each one of those 10000 dimensions goes uphill in either direction. It's overwhelming likely that there's at least one way out - there are many many different ways to set the params of the net for each function. Permutation is a simple example.

We really have no idea how these things work.

Anyone who tells you otherwise is lying to you...

Re: AlphaGo Zero: Learning from scratch

#186

Post by xoroshiro »

EvgeniyZh wrote: post #41
Florin_Andrei wrote: post #25

Earlier quoted context omitted.

I think chess is less compelling because, in a sense, it is a "solved problem" - superhuman AI chess players already exist. And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.

Well, I'd love to see NN solution beating top chess engines. It might also introduce novelty to the game, just as regular engines did

I have some idea how it MIGHT work, but it would be a very boring solution involving 'learning' Stockfish's parameters and HOPING to find improvements to something like integrating time management and search/pruning into it.

I wouldn't bet on it though. SMP is notoriously hard to work with alpha-beta search and there are a lot of clever tricks (which is probably still not perfect). Maybe with ASICs, you could make it stronger, but then it wouldn't be as fair a comparison.

Re: AlphaGo Zero: Learning from scratch

#187

Post by andbberger »

madez wrote: post #184
andbberger wrote: post #175

Earlier quoted context omitted.

Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...

If he changes his opinion, which I understand to be models of the brain in this case, and each iteration improves the model, then that is perfectly fine. It would be bad if someone did not change their view in case of inconsistent evidence.

For political opinions sure, but if he's changing his opinions so often ...

When you're a big scientific figure, I think that you have some extra responsibility to the public to only say things you're very confident about. Or otherwise very clearly communicate your uncertainty!!

Re: AlphaGo Zero: Learning from scratch

#188

Post by xelxebar »

alex_stoddard wrote: post #114
kndyry wrote: post #104

Earlier quoted context omitted.

I don't think it's an overstatement to say that, since playing Lee Sedol in 2016, AlphaGo has completely revolutionized professional and amateur go. It's certainly not unprecedented — the last major revolution happened in the early 20th century (often called the 'Shin Fuseki' era [0]) — but AlphaGo has demonstrably surpassed any previous high-water mark. > I wonder if this system produced more new styles of play. Abs…

I have only skimmed the paper but one thing I don't see any discussion of is whether komi (the handicap given to white for going second) is correct. They do say the rules used for all games, including self-play, set komi consistently to 7.5 . If the strongest AI was consistently winning predominantly with one color it would be an indication that komi isn't fair for the best play. Of the 20 games released for the stro…

Michael Redmond mentions this in the AlphaGo vs AlphaGo review series he's doing with the AGA. AlphaGo selfplay games are with 7.5 komi under Chinese rules, and apparently, Deepmind has stated that black vs white wins is almost exactly 50/50. IIRC Redmond mentioned that white (?) only had some sub 1% advantage in the entire self-play corpus.

Re: AlphaGo Zero: Learning from scratch

#189

Post by andai »

skinner_ wrote: post #80
Jyaif wrote: post #4

"It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features." This is amazing! The technology they came up with must be super generic.

Also, unsupervised. Also, no rollouts. They got rid of a lot of complexity. At this point it looks like a reasonable challenge to write a superhuman Go AI in 500 lines of unobfuscated python.

I was wondering about this: can we study AlphaGo Zero and other nets created in the same way for similarities, extract and study them? Or are we limited to observing the behavior and learning from that?
Post reply on HN