Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

151–160 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#151

Post by btilly »

gizmo686 wrote: post #121
alex_stoddard wrote: post #114

Earlier quoted context omitted.

I have only skimmed the paper but one thing I don't see any discussion of is whether komi (the handicap given to white for going second) is correct. They do say the rules used for all games, including self-play, set komi consistently to 7.5 . If the strongest AI was consistently winning predominantly with one color it would be an indication that komi isn't fair for the best play. Of the 20 games released for the stro…

You can only change komi by full point increments. There is a .5 to break ties, but a komi of 7.5 is identical to one of 7.4. From a theoretical standpoint, any non-integer komi should lead to one player winning 100% of the time. So even if the actual win ratio is 14:6 at komi=7.5 that might still be the best value.

If you had an estimate of the real difference, you could switch to breaking ties randomly. Black wins 60% of the ties, white wins 40%. There will be a ratio at which each side should win 50% of the time.

I agree that with perfect play, it will be a 50% of a tie to each side. But it is still interesting to ask for a better estimate of practical play.

Re: AlphaGo Zero: Learning from scratch

#152

Post by CJefferson »

red75prime wrote: post #81
Arcsech wrote: post #20

Earlier quoted context omitted.

While this is promising, there's a long way to go between this and the other things you mentioned. Go is very well-defined, has an unequivocal objective scoring system that can be run very quickly, and can be simulated in such a way that the system can go through many, many iterations very quickly. There's no way to train an AI like this for, say, health: We cannot simulate the human body to the level of detail that'…

> most of the problems required for a general AI aren't well-defined. Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem? > Producing a definitive, objective score for a paper clip collection is very difficult if not impossible. Erm, producing of objective comparison of relative values of Go board positions is still not possible.

The difference is a go AI can play billions of games and a simple 20 line C program can check, for each game, who won.

For "cat in the picture", every picture must have the cat first identified by a person, so the training set is much smaller, and Google can't throw GPUs at the problem.

Re: AlphaGo Zero: Learning from scratch

#153

Post by conistonwater »

Jach wrote: post #127
alex_stoddard wrote: post #114

Earlier quoted context omitted.

I have only skimmed the paper but one thing I don't see any discussion of is whether komi (the handicap given to white for going second) is correct. They do say the rules used for all games, including self-play, set komi consistently to 7.5 . If the strongest AI was consistently winning predominantly with one color it would be an indication that komi isn't fair for the best play. Of the 20 games released for the stro…

I don't remember where I read it but in some earlier versions of AlphaGo they tried a komi of 6.5 and black ended up winning more often. That indicates the correct komi value is 7, but since Go doesn't have ties, you have to pick which side you want to favor to break the tie. (White seems reasonable.)

That doesn't sound right because in Chinese rules, which is what AG uses, komi only changes in steps of two. Are you sure about 6.5? Could it be 5.5?

Re: AlphaGo Zero: Learning from scratch

#154

Post by autarch »

aeleos wrote: post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

> things like Chinese starts that are specific to certain cultures

While it's true that there are national styles of play, the Chinese opening is not called that because it's really popular among Chinese people. It's called that because a particular Chinese pro helped popularize it, even though it was invented by a Japanese amateur.

See https://en.wikipedia.org/wiki/Chinese_opening for some more info. FWIW, I (a caucasian American) use this opening all the time. It's just a generally good opening if you like a certain style of play.

Re: AlphaGo Zero: Learning from scratch

#155

Post by mannigfaltig »

Houshalter wrote: post #141
andbberger wrote: post #98

Earlier quoted context omitted.

Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best. Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convn…

As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.

> It's an uninteresting reduction since linear algebra can describe almost everything.

The question is whether it can do so efficiently. As far as I know, alternating applications of affine transforms and non-linearities are not so useful for some computations that are known to occur in the brain such as routing, spatio-temporal clustering, frequency filtering, high-dimensional temporal states per neuron etc.

Re: AlphaGo Zero: Learning from scratch

#156

Post by conistonwater »

Extended data figure 2 contains something really cool: how long it takes for AG0 to discover a joseki, and how long it takes it to later discard it as non-jokeki. So you could, in theory, evaluate a human joseki by plotting AG0's probability of playing it against training time (or against its Elo rating). What's also cool is that the differences that cause it stop playing a joseki must be really minute, but it can see them anyway.

Re: AlphaGo Zero: Learning from scratch

#158

Post by goatlover »

SubiculumCode wrote: post #90
hetspookjee wrote: post #83

Amazing results, though I am somewhat frightened by how generic this model is and how it achieved such amazing results. I can't help but think that these same techniques can be used to learn how humans react in certain situaties and how they can be, very subtely, be worked to think in a certain way - one that fits the agenda of whatever party is behind it. With the mass surveillance that is Google it's quite doable t…

I'd only worry if it can outperform humans when there are not rules per se. That is, if I put a queen down in the GO board, and start knocking off stones, moving three times a turn, then take a lighter and burn the go board, the AI responds by decapitating my head.

Ha! I do wonder about using a board game where the rules periodically change in simple ways at random. A human could easily adapt to the rule changes while playing and adjust their strategy accordingly. Would a Deep Learning algorithm be able to do this?

If we keep the board and pieces digital, then the board could change shape, the pieces could change color indicating a random association with a rule change, and what not.

Re: AlphaGo Zero: Learning from scratch

#159

Post by pmontra »

edanm wrote: post #105
jasonwatkinspdx wrote: post #100

Earlier quoted context omitted.

> I wonder if this system produced more new styles of play. One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

I'm not 100% sure I agree. It values probability of victory because that's it's goal. For humans, aiming only for probability of victory might not be as good, because we're much worse at estimating probabilities. So aiming for maintaining a large margin at all times is conceivably the best proxy that we can use in practice .

Agreed. I know I'm winning by 4 points but I have no idea about my probability of winning. However if I'm winning I know that I should play low risk moves and refrain from starting complicated fights. That increases the probability of winning. IMHO the exact value is out of reach for human beings.
Post reply on HN