Live data from Hacker News

AlphaGo beats Lee Sedol 3-0 [video]

youtube.com

401–410 of 428 posts

Re: AlphaGo beats Lee Sedol 3-0 [video]

#401
post #82

Earlier quoted context omitted.

Challenging Ke Jie is way too small a goal for DeepMind at this point. I wonder if even the idea from the AGA stream today, to get all the best pros in the world together and challenge AlphaGo as a team, is enough. Perhaps releasing the core AlphaGo as open source (to the extent it's not dependent on internal Google machinery), or at least publishing its trained model, may be the next step. Let people "challenge them…

I really hope they bother with one more match at least. It was a pity human vs computer idea basically died in chess after very unconvincing win by Deep Blue (even though it lost 2 previous matches). Humans had at least a few years more of good resistance back then. I can think of a worthy goal for AlphaGo: make a program which can play better than top pros which runs on a macbook pro.

Closer to their goal: Make a generalized AlphaGo program that runs on a computer no heavier than 3.3 lbs and uses less than 20 watts of power. Macbook pro is currently 30 watts and 4.5 lbs. So, that's pretty close. But the parent company, Google DeepMind, doesn't have go playing as their ultimate goal.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#402
post #57

In a recent interview [1], Hassabis (DeepMind founder) said they'd try training AlphaGo from scratch next, so it learns from first principles. Without the bootstrapping step of "learn from a database of human games", which introduce human prejudice. As a Go player, I'm really excited to see what kind of play will come from that! [1] http://www.theverge.com/2016/3/10/11192774/demis-hassabis-in...

I believe the whole point of pretraining on reference policies, which a collection of "optimally" played human games is, is just avoidance of bad local optimum.

It can be a case that training and learning on just a learned policy is going to get you stuck in a local optimum that is of worse quality than the one with pretraining.

If they stored all of the AI played games their reference policy (the data) would be of extreme value. You could train a recurrent neural network, without any reinforcement learning, that you could probably run on a smartphone and beat all of the players. You wouldn't need a monte carlo search too.

There are algorithms [1] that have mathematical guarantees of achieving local optimality from reference policies that might not be optimal, and can even work better than the reference policy (experimentally) - assuming that the reference policy isn't optimal. The RNN trained with LOLS would make jointly local decisions over the whole game and each decision would guarantee that a minimization of future regret is being done. Local optimality mentioned here isn't finding a locally optimal model that approximates the strong reference policy, it means that it will find the locally optimal decisions (which piece to put where) without the need for search.

The problem is that for these algorithms you have to have a closely good reference policy, and given a small amount of human played Go games, reinforcement learning was the main algorithm instead, it allowed them to construct a huge number of meaningful games, from which their system learned, which allowed them to construct a huge number of more meaningful games, etc.

But, now when they have games that have a pretty good (AlphaGo is definitely playing on a superhuman level) reference policy, they can train the model based on that reference policy and they wouldn't need a search part of the algorithm at all.

The model would try to approximate the reference policy and would definitely be worse than AlphaGo real-search based policy, but it wouldn't be significantly worse (mathematical guarantee). The model is trained starting from a good player, and it tries to approximate the good player, on the other hand, reinforcement learning starts from an idiot player, and tries to become a good player, reinforcement learning is thus much much harder.

[1]: http://www.umiacs.umd.edu/~hal/docs/daume15lols.pdf

Re: AlphaGo beats Lee Sedol 3-0 [video]

#403

Earlier quoted context omitted.

AlphaGo has also improved very quickly. Without doubt, the AlphaGo seen playing against Fan Hui would have lost against Lee Sedol. But in a couple of months its playing level raised significantly. Lee Sedol said he could beat AlphaGo, based on the Fan Hui games. Ke Jie said he could beat AlphaGo, based on the Lee Sedol games. Ke Jie belongs to a similar category than Lee Sedol, and we could see how Lee Sedol was comp…

He said with the same conditions, his chances are 40:60 in favor of him winning.

That would be about right considering Ke Jie is 8-2 vs Lee Sedol and assuming Lee wins one. That assumes that we are seeing AlphaGo's strongest game and not its "just past the current opponent" game. If Lee doesn't win any then it is very difficult to estimate AlphaGo's strength from the games. If Lee wins one game then a 50-50 AlphaGo vs Ke Jie would be expected.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#404
post #210

Earlier quoted context omitted.

Is it possible that all of AlphaGo's strength is in these baffling moves? If humans played enough games against AlphaGo and discovered how to counter the baffling moves, is it plausible that AlphaGo's strength would be lost?

Watching the game and the commentary there's an eerie sensation that AlphaGo is just going along playing the petty local games with Lee Sedol. It's very anthropocentric, but you could imagine AlphaGo saying "I could have crushed you from the start ... but I'll just play along and do that one move to give you the illusion that it was close". But maybe it could have 40 of those moves to play an incredibly confusing gam…

It was stated by one of the developers that AlphaGo doesn't play to maximize the number of points it wins by. It's satisfied to win by one point. It maximizes the probability of winning. It plays thick moves and tries to simplify the board.

If the player forces it to a fight, it will fight back, but that's only because losing the fight will definitely lose the game.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#405
post #384
post #345

Earlier quoted context omitted.

Also, it should be noted that in Go, we're trying to beat the BEST human player, and have done so. In driving, we're just trying to be "good enough" or safe enough -- it doesn't have to be the safest driver in the world. Beating the average human Go player was probably accomplished decades ago, whereas it's not even clear if we're safer than the average human driver (under all conditions). These tasks are just wildly…

Sort of a tangent from the thread: I get the point about "good enough" at the moment, but I wonder if car AI really does need to perform much safer than any human driver before truly autonomous vehicles should be allowed to see widespread adoption. I'm thinking about the difficult problems re: legal and moral responsibility for human written/guided/trained programs like car AI. As well as the fact that, unlike in Go,…

AI cars could be safer now in most cases by simply not doing dumb illegal stuff.

The real problem is dealing with all the edge cases. Think of this edge case. You pull up to a red light, a guy with a gun starts running at your car in a manner you perceive to be threatening.

You as a human are most likely going to step on the gas and get the hell out of there saving yourself, at some risk of causing a traffic accident.

The car will just sit there till the light turns green while the windows get shot out and you get dragged out of the car.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#406
post #387
post #302

Earlier quoted context omitted.

I think from a technological perspective there's very little question that AlphaGo could play ko. I would have imagined that AlphaGo would be better at ko than most human players since it's a question of balancing risk across the entire board. Human players might be more likely to be exhausted and choose suboptimally by the calculation deciding between different stakes on the board, but MCTS will correctly optimize f…

Don't have the link offhand, but I read on Reddit that Lee Seedol and a few other Go professionals pulled an all-nighter coming up with ways to beat AlphaGo, and one of their guesses was that AlphaGo would be bad at Ko's. I think the reasoning was because that hadn't happened in the games, so they assumed it was avoiding them.

>so they assumed it was avoiding them.

I think us humans made a critical error in that line of thinking.

It didn't avoid ko because of the risk of loss.

It avoided ko because of the lack of strategic win.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#407
post #74

Earlier quoted context omitted.

The great go champion Otake Hideo famously said that if he were to play go against God Himself, he would take only a three stone handicap, and if his life depended on it, he would take four. Alphago's not perfect, but it would still be very interesting to watch such a handicapped game.

Can it even play a handicapped game? I'm assuming the handicap means the other play can begin with several stones on the board. AlphaGo would have no training on such a configuration and would likely not know how to approach it.

Well the same thing was mentioned about Kos and game three showed that AlphaGo was very effective by both having few Ko threats and playing them when imperative. It also demonstrated effective gameplay as black with 4.4 & 4.3 opening.

Considering the large training data set, handicap games would certainly be included. It's a great way to test aggressiveness and the effectiveness to reduce the opponents' territory whilst building your own.

As an amateur, handicap games where you try to overcome 3 or 4 stones are some of the most educational experiences. It requires a lot more creative thinking and perseverance to come back from such a disadvantage.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#408

Go was the last perfect information game I knew where the best humans outperformed the best computers. Anyone know any others? Are all perfect information games lost at this point? Can we design one to keep us winning?

Why bother? It's working with radically incomplete and imperfect information where the core of human intelligence actually lies.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#409

Earlier quoted context omitted.

The difference between being outdone by another human and being outdone by a computer is that the computer's efforts are nearly infinitely reproducible, given the processing power. So a more apt analogy would be if there was someone inside every cellphone who could write books, papers, or symphonies better than you. That day is coming.

And it would be great. Think of all the great symphonies and books!

And the economic preduction. And the influx of wealth into underdeveloped countries. And all of the people not dying.

Re: AlphaGo beats Lee Sedol 3-0 [video]

#410
post #355

Incredible news. We have all heard all of the positive coverage and how tremendous it is. What I find interesting is that how come nobody is talking about the potential of AlphaGo as a war strategizing AI? If you provide terrain(elevation etc.) information, AlphaGo can be used to corner opponents into an area surrounded by mountains where AlphaGo is sitting on the mountains. We all know what happens after that. Don't…

This is because of ethical reasons: one of the conditions that Deepmind made with Google in their deal was for an "embargo on using its technology for military and intelligence applications."

Source: http://www.theguardian.com/technology/2016/feb/16/demis-hass...

Post reply on HN