AlphaGo obviously made mistakes in game 4 under the pressure from LSD's brilliant play. I'd like to know if the "dumb moves" are caused by the lack of pro data or some more fundamental flaws with the algorithm/methodology. AlphaGo was trained on millions of amateur games, but if Google/Deepmind builds a website where people (including prop players) can play with AlphaGo, it would be interesting to see who improves faster.
Lee Sedol Beats AlphaGo in Game 4
101–110 of 471 posts
Re: Lee Sedol Beats AlphaGo in Game 4
#102Earlier quoted context omitted.
> it seems AlphaGo sort of bugged out and started making wrong moves Didn't they say that it's not considered a "bug" but rather how AlphaGo "thinks"? "when it's winning it doesn't care about how much it's winning, and when it's losing it doesn't care how bad it's losing"
it does seem like it is suboptimal behavior to keep making obviously dumb plays instead of attempting a comeback like a professional human would.
Speculating that the reinforcement learning phase reinforced all the best winning strategies but had few examples of weak positions out of which the AI had to fight.
Re: Lee Sedol Beats AlphaGo in Game 4
#103negativist paranoid skeptic could say that it would be a good move for the team to intentionally make αgo lose single battle in the moment it has already won the war..
Re: Lee Sedol Beats AlphaGo in Game 4
#104So AlphaGo is just a bot after all... Toward the end AlphaGo was making moves that even I (as a double-digit kyu player) could recognize as really bad. However, one of the commentators made the observation that each time it did, the moves forced a highly-predictable move by Lee Sedol in response. From the point of view of a Go player, they were non-sensical because they only removed points from the board and didn't a…
Or maybe it was just the computer version of grasping at straws. None of the future gamestates looked good, so it ended up picking whatever could at least theoretically lead to a comeback, even if that would require Lee Sedol to miss a completely obvious move.
Re: Lee Sedol Beats AlphaGo in Game 4
#105Earlier quoted context omitted.
I don't think we're talking about the same concept. I'm not familiar with the concept of label bias, and the literature I'm familiar with has not referred to label bias as the problem I'm talking about. Also, I'm not sure how a problem with probabilistic graphical models translates to the neural net policies of AlphaGo. I fail to see how a "per-state normalization of transition scores" translates to there being a bia…
Yes, the "label bias" is more of a structured learning / joint learning term that is present in natural language processing. But reinforcement learning suffers only if you do the learning to minimize local loss of the decision (label) - if you try to build a classifier that minimizes its loss on local decisions, instead on sequence of decisions. Their value policy network isn't trained jointly and can compound errors…
Again, I don't think we're talking about the same concept. I also fail to see how training over an entire trajectory is going to help you with trajectories you've never seen. Also, these nets are definitely trained with discounted long-term rewards.
Re: Lee Sedol Beats AlphaGo in Game 4
#106If it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then co…
There is a possibility that the AlphaGo team, knowing that it has already won a decisive victory, wanted to spare the champion a crushing 5-0 defeat and so commanded the AI to lose on purpose.
Nah, they let the european champion lose 5-0.
Re: Lee Sedol Beats AlphaGo in Game 4
#107The game seemed to be going in AlphaGo's favour when it was half way through. Black (AG) had secured a large area on the top that seemed nearly impossible to invade. It was amazing to see how Lee Sedol found the right moves to make the invasion work. This makes me think that if the time for match was three hours instead of two, maybe a professional player will have enough time to read the board deeply enough to find…
The thing is, Lee Sedol didn't "find the right moves"; the wedge at L11 shouldn't have worked. If black's 79th move had been at L10 instead of K10, Sedol would likely have resigned on the spot.
Ah! This matches up with Demis' tweet that black's 79th move was later determined to be a mistake. In MCTS you wouldn't normally go back to moves you've already played unless it's an incomplete information game. However I guess for reinforcement learning (even if it's not actually done during these matches), you would go back and update the estimated values of moves already played, which explains how they know that.
Re: Lee Sedol Beats AlphaGo in Game 4
#108Earlier quoted context omitted.
Do you think Lee could use this as a way to crack AlphaGo?
From my limited knowledge of the game, a few of the moves that Lee made before AlphaGo "lost its mind" were a tad on the aggressive side. The conventional wisdom in Go is to prefer more conservative moves (increasingly so as the game progresses). Usually, if your opponent is being overly aggressive then you want to play more conservatively and wait for them to make a mistake, but in AlphaGo's case, it attempted to ma…
> If I had to guess (and this is pure speculation), AlphaGo has no concept of waiting for its opponent to make a mistake. Instead, it assumes its opponent will continue to make the best possible follow-ups
One of the DeepMind guys just confirmed that this is how AlphaGo operates in the press conference.
Re: Lee Sedol Beats AlphaGo in Game 4
#109In the post-game press conference I think Lee Sedol said something like "Before the matches I was thinking the result would be 5-0 or 4-1 in my favor, but then I lost 3 straight... I would not exchange this win for anything in the world."
Demis Hassabis said of Lee Sedol: "Incredible fighting spirit after 3 defeats"
I can definitely relate to what Lee Sedol might be feeling. Very happy for both sides. The fact that people designed the algorithms to beat top pros and the human strength displayed by Lee Sedol.
Congrats to all!
Re: Lee Sedol Beats AlphaGo in Game 4
#110Earlier quoted context omitted.
According to the head of DeepMind, AlphaGo made a mistake in evaluating move 79: https://twitter.com/demishassabis/status/708928006400581632
> Mistake was on move 79, but #AlphaGo only came to that realisation on around move 87 That's cool to think of AlphaGo having "realizations"
It is one way to say that AlphaGo's value network can get wrong.