Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lee Sedol Beats AlphaGo in Game 4

gogameguru.com

Post reply on HN
Print view View original

151–160 of 471 posts

Re: Lee Sedol Beats AlphaGo in Game 4

#151

Post by meric »

jballanc wrote: post #135
wodenokoto wrote: post #127

Earlier quoted context omitted.

I doubt its loss function tries to maximize the prediction of the opponents next move.

I doubt it's considering that directly. What I'm guessing (without having really studied the paper in depth) is that if you have two board positions which would score identically in the end, but where one has fewer remaining liberties than the other (i.e. one is deeper into the tree), then it will be scored more highly. In other words, the humans commentating the game were evaluating the moves as non-sensical because…

The bigger the game board, the more possible states, the better chance a human player has against the AI. Is this the same heuristic in play here - Play a game with more unknowns, gain an edge?

Re: Lee Sedol Beats AlphaGo in Game 4

#152

Post by lifthrasiir »

comboy wrote: post #120
mikeyouse wrote: post #11

Relevant tweets from Demis; Lee Sedol is playing brilliantly! #AlphaGo thought it was doing well, but got confused on move 87. We are in trouble now... Mistake was on move 79, but #AlphaGo only came to that realisation on around move 87 When I say 'thought' and 'realisation' I just mean the output of #AlphaGo value net. It was around 70% at move 79 and then dived on move 87 Lee Sedol wins game 4!!! Congratulations! H…

I'll risk assumption that somebody from Deepmind team is reading this. Guys, please, publish charts of win prob estimated by alpha go in time during these games. Some heatmap telling which moves did it consider as best for both sides during the games would also be cool, but that's surely more time consuming to prepare. It would be great to be able to have such things for top pro tournaments in the future.

I.e. Something like [1] for every move. I was a bit disappointed to learn that such table is only available for that particluar move at the first reading.

[1] http://www.nature.com/nature/journal/v529/n7587/fig_tab/natu...

Re: Lee Sedol Beats AlphaGo in Game 4

#153

Post by meric »

magoghm wrote: post #89

Right now I don't know if I'm more impressed by AlphaGo's artificial intelligence or its artificial stupidity. Lee Sedol won because he played extremely well. But when AlphaGo was already losing it made some very bad moves. One of them was so bad that it's the kind of mistake you would only expect from someone who's starting to learn how to play Go.

AlphaGo kept making bad moves in such a way the rest of the game becomes more and more predictable - each of the moves Lee Sedol makes could be described as the only obvious one.

On the surface, as an analogy, it sounds like investors in financial markets, capitulating, selling at a loss for a more risky outcome. In hindsight almost always bad moves, but at the time of making them it feels right because it's removing risk. Investors are losing, and then when capitulating they make even worse moves, like selling at market bottoms.

http://www.investopedia.com/terms/c/capitulation.asp?layout=...

Re: Lee Sedol Beats AlphaGo in Game 4

#154

Post by antonioalegria »

throw121221 wrote: post #125
antonioalegria wrote: post #114

Don't want to sound all Conspiracy Theory but somehow this feels planned.. It plays into DeepMind's hand to not have the machine completely trouncing the human. It's less scary and keeps people engaged further into the future. Also seems in-line with the way Demis was "rooting" for the human this time – they already won so now they focus on PR.

Shush! You cannot say anything like this on HN! People here want to believe that the world is fair, Google and Apple do not do evil and pg is the greatest philosopher who has ever lived.

Wow, I think it's not that farfetched. See the thought wasn't appreciated. It's a worthwhile discussion, imho

Re: Lee Sedol Beats AlphaGo in Game 4

#155

Post by Joof »

tux1968 wrote: post #141
jballanc wrote: post #97

Earlier quoted context omitted.

Well, that's where AlphaGo and the progress in Go AI that it represents is so exciting ! The game of Go is so fluid with such a huge number of possible positions that players tend to adopt certain styles of play en masse . I've heard it said that you can identify a Go player's mentor or "house" just by the style of play they use. This has also resulted in larger shifts in playing style over time. Studying very old (a…

AlphaGo will learn from any new styles and apply them effectively without mistakes from fatigue or inattention. This incarnation of AI is not creative, it wont generate new play styles, that is still the domain of top human players for now. But it will ruthlessly learn and adopt any new and improved strategies. That's really the point to take away from its success so far.

AlphaGo mostly plays against itself, meaning it learns in a very separate environment. It certainly might come up with novel strategies.

Re: Lee Sedol Beats AlphaGo in Game 4

#156

Post by dllu »

emcq wrote: post #27

That was really cool! It seemed after the brilliant play in the middle the most probable moves for winning required Lee Sedol to make impossibly bad mistakes for a professional, which would be a prior that AlphaGo doesn't incorporate. I've heard the training data was mostly amateur games so perhaps the value/policy networks were overfit? Or maybe greedily picking the highest probability, common with tree search appro…

Failure to generalize is not always caused by overfitting. Even if there is no overfitting, deep neural networks seem to learn a surprisingly discontinuous function. Consequently, in rare cases, they can misclassify things with great confidence. [0]

From the cited paper,

> [Experimental results] suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular model or to the specific selection of the training set.

Anyway, Monte Carlo Tree Search is bad at losing positions. In general, you want to delay the impending catastrophe as long as possible instead of making stupid moves that make your position worse and worse. However, MCTS uses random rollouts to the end of the game, which sometimes make it difficult to ascertain if the inevitable doom is near or far.

Also, MCTS converges very, very slowly and is likely to miss a unique, single winning continuation.

I think it is probably a combination of both AlphaGo's value network failing to realize the good position of Lee Sedol after his brilliant play, and the MCTS failing to spot the unique winning sequence for Lee, that caused it to make the mistake. But we should probably wait for official analysis from the Deepmind team to see what exactly went wrong.

[0] "Intriguing properties of neural networks" http://arxiv.org/pdf/1312.6199.pdf

Re: Lee Sedol Beats AlphaGo in Game 4

#157

Post by antonioalegria »

runesoerensen wrote: post #123
antonioalegria wrote: post #114

Don't want to sound all Conspiracy Theory but somehow this feels planned.. It plays into DeepMind's hand to not have the machine completely trouncing the human. It's less scary and keeps people engaged further into the future. Also seems in-line with the way Demis was "rooting" for the human this time – they already won so now they focus on PR.

Don't post unfounded conspiracy theories if you don't want to sound like a conspiracy theorist.

A lot of comments here are unfounded. It was a thought I had that imo warrants some discussion (which you may disagree)

Re: Lee Sedol Beats AlphaGo in Game 4

#158

Post by jacquesm »

argonaut wrote: post #51

If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…

The same happens to people, especially people that study theory. You can totally throw them off their game by making a non-standard move, even a relatively bad one as long as it breaks their existing pre-conceived notions about how the game should progress.

Of course against a really strong player you're going to get beaten after that but a weak player strong on theory will have a harder time.

Re: Lee Sedol Beats AlphaGo in Game 4

#159

Post by Udik »

diskcat wrote: post #78
argonaut wrote: post #51

If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…

There is a possibility that the AlphaGo team, knowing that it has already won a decisive victory, wanted to spare the champion a crushing 5-0 defeat and so commanded the AI to lose on purpose.

Yesterday I kind of entertained the thought that they could have tried to make Alphago waste a move somewhere at the beginning of the game to give Sedol the equivalent of a stone advantage and see how Alphago could handle that. But it's clear that any move like that would have been immediately detected by the professionals who can read the game as I read the morning newspapers, so no.

Re: Lee Sedol Beats AlphaGo in Game 4

#160

Post by jacquesm »

barosl wrote: post #57
agildehaus wrote: post #39

Earlier quoted context omitted.

AlphaGo resigns The result "W:Resign" was added to the game information. Edit: Tinyyy is right.

According to this picture[1], it is more likely "W+Resign". I'm curious why a plus sign is used instead of a colon! [1] http://gall.dcinside.com/board/view/?id=baduk&no=109200&page...

Bad google+ joke withheld.
Post reply on HN