Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo beats Lee Sedol again in match 2 of 5

gogameguru.com

Post reply on HN
Print view View original

261–270 of 576 posts

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#261

Post by bdalgaard »

ig1 wrote: post #254
fhe wrote: post #239

As someone who studied AI in college and am a reasonably good amateur player, I have been following the matches between Lee and AlphaGo. AlphaGo plays some unusual moves that go clearly against any classically trained Go players. Moves that simply don't quite fit into the current theories of Go playing, and the world's top players are struggling to explain what's the purpose/strategy behind them. I've been giving it…

AlphaGo is essentially built on the work that IBM did on TD-Gammon (a reinforcement learning backgammon player) in the 90s. Pretty much the same thing happened with TD-Gammon with it playing unconventional moves, in the longer term humans ended up adopting some of TD-Gammon's tactics once they understood how they played out, it wouldn't be surprising to see the same happen with Go.

From my understanding, computers have also had this affect on chess. The play styles of younger champions has evolved to the point where unpredictability is actually part of the strategy. I'm not a chess expert by any means, but this quote by Viswanathan Anand (former World Chess Champion) describes it.

  “Top competitors who once relied on particular styles of play are now forced to mix up their strategies, for fear that powerful analysis engines will be used to reveal fatal weaknesses in favoured openings....Anything unusual that you can produce has quadruple, quintuple the value, precisely because your opponent is likely to do the predictable stuff, which is on a computer” [1]
[1] http://www.businessinsider.com/anand-on-how-computers-have-c...

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#262

Post by ap22213 »

fhe wrote: post #239

As someone who studied AI in college and am a reasonably good amateur player, I have been following the matches between Lee and AlphaGo. AlphaGo plays some unusual moves that go clearly against any classically trained Go players. Moves that simply don't quite fit into the current theories of Go playing, and the world's top players are struggling to explain what's the purpose/strategy behind them. I've been giving it…

Sometimes optimal solutions don't make sense to the human mind because they're not intuitive.

For instance, I developed a system that used machine learning and linear solver models to spit out a series of actions to take in response to some events. The actions were to be acted on by humans who were experts in the field. In fact, they were the ones from whom we inferred the relevant initial heuristics.

Everyday, I would get a support call from one of the users. They'd be like, 'this output is completely wrong. You have a bug in your code.'

I'd then have to spend several hours walking through each of the actions with them and recording the results. In every case, the machine would produce recommended actions that were optimal. However, they were rarely intuitive.

In the end, it took months of this back and forth until the experts began to trust the machine outputs.

This is the frightening thing about AI - not only can an AI outperform experts, but it often makes decisions that are incomprehensible.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#263

Post by jcrei »

criddell wrote: post #228
Cookingboy wrote: post #116

Someone somewhere asked why a lot of people in the Go community is taking this in a somewhat hard way, here is my hypothesis: Go, unlike Chess, has deep mytho attached to it. Throughout the history of many Asian countries it's seen as the ultimate abstract strategy game that deeply relies on players' intuition, personality, worldview. The best players are not described as "smart", they are described as "wise". I thin…

> Now imagine the winning author of the next Hugo Award turns out to be an AI, how unsettling would that be. I've been thinking precisely about that. I think a book written by a machine will make the NYT bestseller list within our lifetimes (I would give it a 75% chance within 10 years, but that's just a gut feeling).

But aren't the winners of literary awards sort of subjective? I mean, chess and Go are based on beating a direct opponent based on a set of known rules. But getting an award for the best book that is picked by a judge means that you have to hit all the right notes in all the right places for that specific judge or panel. I'm not saying it won't happen, just that it's more subjective

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#264

Post by losvedir »

ganwar wrote: post #216
mixedmath wrote: post #9

This game was largely played extremely well by both sides. There were a a few peculiar-seeming moves made by AlphaGo that the commentator found very atypical. These moves ended up playing a very important role in the end game. I should also say that it's somewhat clear that Sedol made one suboptimal move, and AlphaGo capitalized on it. Interestingly, the English commentator made the same mistake as he was predicting…

> I should also say that it's somewhat clear that Sedol made one suboptimal move, and AlphaGo capitalized on it. Can you please indicate this time in the video of the game? Thanks a lot.

I watched the live stream commentated by Myungwan Kim so I know what the parent is referring to (later in the game, Myungwan Kim referred back to that possibly being a mistake), although not being a Go player myself most of it was over my head.

It was fairly early in the game and about 4-5 lines down from the top and towards the center, center-left. Apparently, Lee Sedol played a little conservatively and "didn't take a ko" (?) when he could have.

Hope that helps you figure out what they're referring to, or maybe someone else can chime in, but that's what I remember.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#265

Post by rurban »

pavelrub wrote: post #134
davelondon wrote: post #91

Let's compare Go and Chess. We all know that Go is more complex that Chess, but how much more? There's 10^50 atoms in the planet Earth. That's a lot. Let's put a chess board in each of them. We'll count each possible permutation of each of the chess boards as a separate position. That's a lot, right? There's 10^50 atoms, and 10^40 positions in each chess board so that gives us 10^90 total positions. That's a lot of p…

This doesn't seem to be the main reason why Go is harder than chess for computers. It was noted that even in 9x9 Go, with a comparable branching factor to Chess, traditional Go programs are still no stronger than on big boards. The main difficulty for Go is that it is much harder to evaluate board positions. So in Chess the depth of the search can be significantly reduced by using a reasonable evaluation function, wh…

This evaluation function does exist, and it's better than the super-simple chess evaluation function.

See, a chess program needs to find a lot of valid moves (see Deep Blue which won because it had stupid but extremely fast HW move generators), evaluate the moves and do a very deep search, up to 14, out of the very few alternatives. Russian chess programmers were better those times. They came up with AVL trees e.g. But hardware won.

In Go it's completely different. A move generator makes no sense at all, and a depth search of 14 neither. There are not a few alternatives, there are too many. What you need is a good overall pattern matching of areas of interest and an evaluation of those areas. And we saw that this feature outplayed Lee Sedol. Sedol couldn't quite follow in the recalculation of the areas.

Same as in chess AlphaGo learned the easy thing, that the center is more important than the corners, something Lee forgot during the game. But it's not a deep search, it's a very broad search, and very complicated evaluation function. A neural net is perfect for this function.

> whereas in Go no such function seems to exist.

It does exist. It's the neural net. It's a simple pattern recognizer, which learns over time more and more.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#266

Post by thom »

Cookingboy wrote: post #116

Someone somewhere asked why a lot of people in the Go community is taking this in a somewhat hard way, here is my hypothesis: Go, unlike Chess, has deep mytho attached to it. Throughout the history of many Asian countries it's seen as the ultimate abstract strategy game that deeply relies on players' intuition, personality, worldview. The best players are not described as "smart", they are described as "wise". I thin…

I follow neither Go nor martial arts, but there seem to be some interesting parallels here with some emotional reactions to what appears to be the relative weakness of karate or kung fu versus grappling in UFC. The mystical aura of these martial arts as traditionally practised for hundreds of years suddenly falls away in the face of what often seems like brute force.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#267

Post by taneq »

octatoan wrote: post #237
jules wrote: post #98

Earlier quoted context omitted.

It would be great if his co-commentator was a computer scientist who is knowledgeable about AlphaGo's algorithm.

Indeed, I wish someone could talk about how the value/policy thing works.

As I understand it, the value network takes the place of the heuristic for scoring a given board layout, and the policy network takes the place of the heuristic for ordering moves from most to least promising.

When searching the game tree, at each ply the most promising N moves are examined (as determined by the policy network) and leaves of the game tree are scored by the value network.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#268

Post by colanderman »

astrofinch wrote: post #257
fhe wrote: post #239

As someone who studied AI in college and am a reasonably good amateur player, I have been following the matches between Lee and AlphaGo. AlphaGo plays some unusual moves that go clearly against any classically trained Go players. Moves that simply don't quite fit into the current theories of Go playing, and the world's top players are struggling to explain what's the purpose/strategy behind them. I've been giving it…

There's also the fact that some of the unexpected moves were apparently more about solidifying against a loss than increasing the magnitude of a win. Which has its own kind of eerie implication: since AIs (like all computer programs) do what you say, not what you mean, the "intelligent species" can sometimes work really intelligently towards a goal that wasn't quite what you had in mind. (Gets especially interesting…

About a year ago I wrote an AI to play the board game "Hive" (shares some similarities with chess). Because I scored all wins equally, it behaved almost exactly like this. It would simply try to minimize my advantage while always keeping open the possibility for it to win, almost like a cat toying with prey. It never actually would make the winning move – however obvious – until it had no other options!

I fixed this behavior by scoring earlier wins higher than later wins. Now it will actually finish games (and win), but almost invariably its edge is very small, no matter how well or poorly I play. Because of the new win scoring, it willingly sacrifices its own advantage if it means securing a win even one turn earlier. (And since scoring is symmetrical, this has the added advantage of working to delay any win it sees for me, thus increasing the possibility of me making a mistake!)

I suppose I could try modifying the scoring rules again, to weight them by positional advantage. A "show off" mode if you like :) And again, with the flip side of working to create the least humiliating losses for itself.

Re: AlphaGo beats Lee Sedol again in match 2 of 5

#270

Post by thomasahle »

megalodon wrote: post #251
fhe wrote: post #239

As someone who studied AI in college and am a reasonably good amateur player, I have been following the matches between Lee and AlphaGo. AlphaGo plays some unusual moves that go clearly against any classically trained Go players. Moves that simply don't quite fit into the current theories of Go playing, and the world's top players are struggling to explain what's the purpose/strategy behind them. I've been giving it…

I think an important point was brought up by the Google engineer in the beginning of the game: Humans usually consider moves that put them ahead by a greater margin and base their strategies on that, while computers don't have that bias.

Was that in the official livestream, or is there an interview somewhere, where things like these are discussed?
Post reply on HN