Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lee Sedol Beats AlphaGo in Game 4

gogameguru.com

Post reply on HN
Print view View original

71–80 of 471 posts

Re: Lee Sedol Beats AlphaGo in Game 4

#71

Post by jballanc »

ljk wrote: post #58
jballanc wrote: post #56

So AlphaGo is just a bot after all... Toward the end AlphaGo was making moves that even I (as a double-digit kyu player) could recognize as really bad. However, one of the commentators made the observation that each time it did, the moves forced a highly-predictable move by Lee Sedol in response. From the point of view of a Go player, they were non-sensical because they only removed points from the board and didn't a…

Do you think Lee could use this as a way to crack AlphaGo?

From my limited knowledge of the game, a few of the moves that Lee made before AlphaGo "lost its mind" were a tad on the aggressive side. The conventional wisdom in Go is to prefer more conservative moves (increasingly so as the game progresses). Usually, if your opponent is being overly aggressive then you want to play more conservatively and wait for them to make a mistake, but in AlphaGo's case, it attempted to match Lee's aggressiveness move-for-move, and Lee was able to capitalize.

If I had to guess (and this is pure speculation), AlphaGo has no concept of waiting for its opponent to make a mistake. Instead, it assumes its opponent will continue to make the best possible follow-ups, and so AlphaGo feels overly compelled to "keep up". In this case, that did it in.

If this is what happened, then yes, I would expect Lee to be able to capitalize.

Re: Lee Sedol Beats AlphaGo in Game 4

#73

Post by mrgordon »

tomlock wrote: post #63
teling wrote: post #33

Earlier quoted context omitted.

I would expect them to be running on Google's distributed infrastructure which has error corrections through end to end checksums

Actually they were running on a single PC! https://twitter.com/demishassabis/status/708489093676568576 ed: oops I misread!

"Using distributed for match but single machine AG very strong also"

Doesn't this imply they weren't using a single PC?

Re: Lee Sedol Beats AlphaGo in Game 4

#74

Post by adnzzzzZ »

keypusher wrote: post #62
adnzzzzZ wrote: post #6

According to the commentary of both streams I was watching, after losing an important exchange in the middle (apparently move 79 https://twitter.com/demishassabis/status/708928006400581632 ) it seems AlphaGo sort of bugged out and started making wrong moves on an already dead group on the right side of the board. After that it kept repeating similar mistakes until it resigned a lot of moves after. But the game was al…

I would avoid thinking of this like a traditional computer program that just "bugged out" due to a glitch or a problem in the software. More accurately, it failed to account for the implications of a move on the board and therefore focused its attention in the wrong place. This happens often in games, I can imagine that a chess master playing an amateur might move his knight or bishop into a position that does immedi…

I mean that it bugged out because the moves it made after missing the exchange in the middle were moves that were obviously wrong, not in a "maybe it's up to something" kind of way, but in an objectively 100% bad kind of way. Even if you can't analyze all possibilities AlphaGo made a number of moves that made absolutely no sense at all even to way lower level players.

Re: Lee Sedol Beats AlphaGo in Game 4

#75

Post by techdragon »

mikeyouse wrote: post #11

Relevant tweets from Demis; Lee Sedol is playing brilliantly! #AlphaGo thought it was doing well, but got confused on move 87. We are in trouble now... Mistake was on move 79, but #AlphaGo only came to that realisation on around move 87 When I say 'thought' and 'realisation' I just mean the output of #AlphaGo value net. It was around 70% at move 79 and then dived on move 87 Lee Sedol wins game 4!!! Congratulations! H…

I wish some of these had been re-tweeted by the @DeepMindAI account. I'll definitely be keeping an eye on Demis' account during game 5.

Re: Lee Sedol Beats AlphaGo in Game 4

#76

Post by argonaut »

kotach wrote: post #68
argonaut wrote: post #51

If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…

What you are talking about here is called "label bias". [2] It is present only if training is done badly. When you have a game of Go, or Super Mario level. You don't want to make your decisions by just checking the local features and doing them, because it can be the case that by compounding errors you end up in a state you never saw, and all of the future decisions won't be good. One can avoid these situations by tr…

I don't think we're talking about the same concept. I'm not familiar with the concept of label bias, and the literature I'm familiar with has not referred to label bias as the problem I'm talking about. Also, I'm not sure how a problem with probabilistic graphical models translates to the neural net policies of AlphaGo.

I fail to see how a "per-state normalization of transition scores" translates to there being a bias in value networks towards states with fewer outgoing transitions.

Re: Lee Sedol Beats AlphaGo in Game 4

#77

Post by semiel »

ljk wrote: post #58
jballanc wrote: post #56

So AlphaGo is just a bot after all... Toward the end AlphaGo was making moves that even I (as a double-digit kyu player) could recognize as really bad. However, one of the commentators made the observation that each time it did, the moves forced a highly-predictable move by Lee Sedol in response. From the point of view of a Go player, they were non-sensical because they only removed points from the board and didn't a…

Do you think Lee could use this as a way to crack AlphaGo?

Step 1 of exploiting this is "play a brilliant move to win a fight in the middle game", which is not the easiest thing to repeat.

Re: Lee Sedol Beats AlphaGo in Game 4

#78

Post by diskcat »

argonaut wrote: post #51

If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…

There is a possibility that the AlphaGo team, knowing that it has already won a decisive victory, wanted to spare the champion a crushing 5-0 defeat and so commanded the AI to lose on purpose.

Re: Lee Sedol Beats AlphaGo in Game 4

#79

Post by argonaut »

diskcat wrote: post #78
argonaut wrote: post #51

If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…

There is a possibility that the AlphaGo team, knowing that it has already won a decisive victory, wanted to spare the champion a crushing 5-0 defeat and so commanded the AI to lose on purpose.

Incredibly unlikely.

Re: Lee Sedol Beats AlphaGo in Game 4

#80

Post by nitrobeast »

keypusher wrote: post #40

The crucial play here seems to have been Lee Seedol's "tesuji" at White 78. From what I understand this phrase in Go means something like "clever play" but is something like sneaking up on your opponent with something that they did not see coming. Deepmind CEO confirmed that the machine actually missed the implications of this move as the calculated win percentage did not shift until later. https://twitter.com/demish…

AlphaGo was in over time in game 2.
Post reply on HN