Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

101–110 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#101

> The system starts off with a neural network that knows nothing about the game of Go. It then plays games against itself I might have missed this, but: Where are the actual rules of Go encoded? Mustn't there be some enumeration of what constitutes "capturing," how the win condition of the game is calculated, correct?

Yes, I assume you start with an objective function ("winning Go") that includes the rules.

Re: AlphaGo Zero: Learning from scratch

#102
post #88

Earlier quoted context omitted.

That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence. EDIT: changed a set of neurons to neural network per andbbergers comments

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

Please don't refer to them as black boxes. The internals are fully observable.

Re: AlphaGo Zero: Learning from scratch

#103
post #20
post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

While this is promising, there's a long way to go between this and the other things you mentioned. Go is very well-defined, has an unequivocal objective scoring system that can be run very quickly, and can be simulated in such a way that the system can go through many, many iterations very quickly. There's no way to train an AI like this for, say, health: We cannot simulate the human body to the level of detail that'…

> We cannot simulate the human body to the level of detail that's required

A-ha! So we use AGI for this! :-)

Re: AlphaGo Zero: Learning from scratch

#104
post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

I don't think it's an overstatement to say that, since playing Lee Sedol in 2016, AlphaGo has completely revolutionized professional and amateur go. It's certainly not unprecedented — the last major revolution happened in the early 20th century (often called the 'Shin Fuseki' era [0]) — but AlphaGo has demonstrably surpassed any previous high-water mark.

> I wonder if this system produced more new styles of play.

Absolutely. One such innovation has been the use of early 3-3 invasions [1]. There are many more, and indeed AlphaGo's games are still being analyzed by professional players. Michael Redmond, a 9-dan professional, has been working with the American Go Association on one such series [2].

> I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that are somehow inherent to the game...

Interestingly, yes. Strong players have commented that AlphaGo seems to agree with things that players like Go Seigen [3] have suggested in the past, but that were never fully developed or understood [4].

Very, very interesting work indeed.

[0] https://senseis.xmp.net/?ShinFuseki

[1] https://www.eurogofed.org/index.html?id=127

[2] http://www.usgo.org/news/category/go-news/computer-goai/mast...

[3] https://senseis.xmp.net/?GoSeigen

[4] https://lifein19x19.com/forum/viewtopic.php?f=13&t=14129

Re: AlphaGo Zero: Learning from scratch

#105
post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

> I wonder if this system produced more new styles of play. One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

I'm not 100% sure I agree. It values probability of victory because that's it's goal. For humans, aiming only for probability of victory might not be as good, because we're much worse at estimating probabilities. So aiming for maintaining a large margin at all times is conceivably the best proxy that we can use in practice.

Re: AlphaGo Zero: Learning from scratch

#106
post #102

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

Please don't refer to them as black boxes. The internals are fully observable.

It's a conceptual black box. There's no way for us to understand what each individual neuron is doing.

Re: AlphaGo Zero: Learning from scratch

#107
post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

> I wonder if this system produced more new styles of play. One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

Part of that could be trying to compensate for counting skills that aren't quite at machine levels.

Re: AlphaGo Zero: Learning from scratch

#108
post #102

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

Please don't refer to them as black boxes. The internals are fully observable.

Is there meaningful information in what one observes?

Re: AlphaGo Zero: Learning from scratch

#109
post #102

Earlier quoted context omitted.

Please don't refer to them as black boxes. The internals are fully observable.

It's a conceptual black box. There's no way for us to understand what each individual neuron is doing.

Indeed?

https://arxiv.org/pdf/1312.6034.pdf

Re: AlphaGo Zero: Learning from scratch

#110

> The system starts off with a neural network that knows nothing about the game of Go. It then plays games against itself I might have missed this, but: Where are the actual rules of Go encoded? Mustn't there be some enumeration of what constitutes "capturing," how the win condition of the game is calculated, correct?

If you give a list of "possible next states" from any given state, that should encompass all the rules. If there's a rule that's preventing you from doing something, it's not a possible state.

By nothing they mean hints about what constitutes "good Go strategy". But it implicitly knows all of the rules of go.

Post reply on HN