https://twitter.com/StatsmanBruno/status/708941065877737472/...
[deleted]
Lee Sedol Beats AlphaGo in Game 4
81–90 of 471 posts
Re: Lee Sedol Beats AlphaGo in Game 4
#82That was really cool! It seemed after the brilliant play in the middle the most probable moves for winning required Lee Sedol to make impossibly bad mistakes for a professional, which would be a prior that AlphaGo doesn't incorporate. I've heard the training data was mostly amateur games so perhaps the value/policy networks were overfit? Or maybe greedily picking the highest probability, common with tree search appro…
That's not the definition of overfitting.
If the value/policy model is predictive with a dataset containing only amateur games, but fails to generalize to unseen data with professional games, that seems like a case of overfitting to a dataset only containing amateur games. In this case the expected value network may be different for amateur games than professional games.
Is there something I'm missing?
Re: Lee Sedol Beats AlphaGo in Game 4
#83I was not expecting Lee Sedol to come back and win a game after his first three losses. AlphaGo seemed to be struggling at the end of the match.
I was hoping Lee Sedol would be able to at least win one, humans can 'learn fast'. We cant learn 24/7 in parallel like a computer, but we do seem to have quite the talent at learning 'situational' things very quickly, quite likely because our very survival depended on it in the deep distant past. :-)
I don't believe AlphaGo had the time to do any additional training between matches. So effectively Lee has the ability to 'learn his opponent' while AlphaGo cannot until the entire match set is over because of how long it would take do do additional training.
Re: Lee Sedol Beats AlphaGo in Game 4
#84Montecarlo bots behave weirdly when losing.
Re: Lee Sedol Beats AlphaGo in Game 4
#85https://twitter.com/StatsmanBruno/status/708941065877737472/...
That got me for a second hehe (it's a fake)
Re: Lee Sedol Beats AlphaGo in Game 4
#86Re: Lee Sedol Beats AlphaGo in Game 4
#87Earlier quoted context omitted.
What you are talking about here is called "label bias". [2] It is present only if training is done badly. When you have a game of Go, or Super Mario level. You don't want to make your decisions by just checking the local features and doing them, because it can be the case that by compounding errors you end up in a state you never saw, and all of the future decisions won't be good. One can avoid these situations by tr…
I don't think we're talking about the same concept. I'm not familiar with the concept of label bias, and the literature I'm familiar with has not referred to label bias as the problem I'm talking about. Also, I'm not sure how a problem with probabilistic graphical models translates to the neural net policies of AlphaGo. I fail to see how a "per-state normalization of transition scores" translates to there being a bia…
Yes, the "label bias" is more of a structured learning / joint learning term that is present in natural language processing. But reinforcement learning suffers only if you do the learning to minimize local loss of the decision (label) - if you try to build a classifier that minimizes its loss on local decisions, instead on sequence of decisions.
Their value policy network isn't trained jointly and can compound errors. There are approaches with deep neural networks that don't have a joint training but work pretty well. The reason is that networks have a pretty good memory/representation and by that they avoid much of the problems. But for huge games like Go it is quite possible that more games need to be played for these non-structured models to work well.
Re: Lee Sedol Beats AlphaGo in Game 4
#88Earlier quoted context omitted.
Actually they were running on a single PC! https://twitter.com/demishassabis/status/708489093676568576 ed: oops I misread!
"Using distributed for match but single machine AG very strong also" Doesn't this imply they weren't using a single PC?
They're using the distributed version for these matches, this question was just asked in the post match press conference.
Re: Lee Sedol Beats AlphaGo in Game 4
#89Right now I don't know if I'm more impressed by AlphaGo's artificial intelligence or its artificial stupidity.
Lee Sedol won because he played extremely well. But when AlphaGo was already losing it made some very bad moves. One of them was so bad that it's the kind of mistake you would only expect from someone who's starting to learn how to play Go.
Re: Lee Sedol Beats AlphaGo in Game 4
#90Am I right by asumming, that if they would play another game (AlphaGo black and Lee Sedol white), that Lee Sedol could pressure AlphaGo into makeing the same mistake again?
Could the AlphaGo developers make modifications before the next game to prevent this happening again?
They stated before game 3 that AG had been locked down a while before these games to test the code for bugs and issues. I also think they don't want some of the implications associated with DeepBlue where people modified code even during the games.