Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo beats the world champion Lee Sedol in first of five matches

twitter.com

Post reply on HN
Print view View original

471–480 of 596 posts

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#471

Post by CamperBob2 »

riffraff wrote: post #464
CamperBob2 wrote: post #437

Earlier quoted context omitted.

Remember that "solve computer vision" was considered a summer project. And they're pretty much there. Have you seen some of the latest results in that field?

I think you misunderstood, I am referring to the fact that Marvin Minsky in 1966, asked Gerald Sussman to "spend the summer linking a camera to a computer and getting the computer to describe what it saw". We certainly got nearly there, but it was nearly 50 years later, not 3 months. Similarly, something that might look somewhat simple to us right now, might also be a lot more difficult.

I see what you mean -- sorry, I thought you were referring to a Summer of Code project.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#472

Post by ogrisel »

argonaut wrote: post #168
ogrisel wrote: post #136

Earlier quoted context omitted.

You can do math with continuous and infinite dimensional spaces.

And? This does not address my argument that the complexity is beyond-combinatorially explosive (infinite spaces). I'm not talking about the space of possible board states. I'm talking about merely the set of all possible actions. EDIT: clarified my language to address below reply.

...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy:

http://arxiv.org/abs/1509.02971

The fact that the (reinforcement) learning problem is hard or not is not directly related to whether the observation and action spaces are discrete or continuous.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#473

Post by rictic »

gcr wrote: post #409
hosh wrote: post #370

Earlier quoted context omitted.

The 2003 match was a brute force approach. AlphaGo's architecture resembles much closer to how humans think and learn. I initially learned Go to be able to have some chance of an AI. I then had some transformative experiences that coincided with my early kyu learning of basic Go lessons. On of the big lessons in Go is to learn how to let go of something. Taking solace in anything on the Go board is one of the blocks…

This isn't about Moore's Law though. From the AlphaGo paper: > During the match against Fan Hui, AlphaGo evaluated thousands of times > fewer positions than Deep Blue did in its chess match against > Kasparov; compensating by selecting those positions more intelli- > gently, using the policy network, and evaluating them more precisely, > using the value network—an approach that is perhaps closer to how > humans play.…

My understanding is that it is much more expensive for AlphaGo to evaluate a position than it was for Deep Blue. I'm not certain, but I would be surprised if AlphaGo did not need significantly more computation than Deep Blue.

edit: some actual estimates. Deep Blue had 11.38 GFLOPS[1]. According to the paper in Nature, distributed AlphaGo used 1202 CPUs and 176 GPUs. A single modern GPU can do between 100 and 2000 double precision GFLOPS[2]. So from GPUs alone AlphaGo had access to 4-5 orders of magnitude more computing power than Deep Blue did.

1] https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)

2] https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#474

Post by ogrisel »

ogrisel wrote: post #472
argonaut wrote: post #168

Earlier quoted context omitted.

And? This does not address my argument that the complexity is beyond-combinatorially explosive (infinite spaces). I'm not talking about the space of possible board states. I'm talking about merely the set of all possible actions. EDIT: clarified my language to address below reply.

...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy: http://arxiv.org/abs/1509.02971 The fact that the (reinforcement) learning problem is hard or not is not directly related to whether the observation and action spaces are…

see also by the same team:

http://arxiv.org/abs/1602.01783

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#475

Post by LockeWatts »

zouhair wrote: post #436
cm2012 wrote: post #429

After Go, the next AI challenge they're looking at is Starcraft: https://twitter.com/deeplearning4j/status/706541229543071745

Good luck with that.

Starcraft in many ways is a much easier game for an AI to beat top pros at than Go.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#476

Post by cgearhart »

j2kun wrote: post #460
cgearhart wrote: post #330

Earlier quoted context omitted.

> What? Exactly They used the game database to learn the value network, then reinforcement learning of the policy network was performed on self-play games. I.e., the machine learned to play from existing data, then played against itself to learn the search heuristics (the policy network) without the need for expert data.

Your claim still doesn't make sense. They either used expert data or they didn't. If the algorithm would lose when they remove the expert data, then they really do need expert data. The tree search wasn't even the novel part of the algorithm... the authors even cite others who had used the identical technique in previous Go algorithms.

It seems that my original comment is unclear. My apologies for the ambiguity. I did not mean that they do not need any expert data, but that part of the training did not require a training data set.

They definitely need training data to learn the value function, but training the policy network is based on self-play. While MCTS is not new, I believe bootstrapping reinforcement learning with self-play to train a policy network that guides the MCTS is novel.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#477

Post by hosh »

nsajko wrote: post #458
hosh wrote: post #403

Earlier quoted context omitted.

Yep. I was just talking with the founder of a startup I work with. His son was born in the past 5 months or so. The son is never going to live in a world that doesn't have deep learning. Like the kids who never knew what the world was like before the smartphone. Like the kids who never knew what the world was like before the web browser. And AI is just one strand. There are several strands that are as deeply changing…

The son is never going to live in a world that doesn't have deep learning. Except if a big solar flare hits us ;}

Or a bunch of other things. Maybe our civilization collapses from peak oil or something.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#478

Post by LockeWatts »

sago wrote: post #469
cm2012 wrote: post #429

After Go, the next AI challenge they're looking at is Starcraft: https://twitter.com/deeplearning4j/status/706541229543071745

The obvious problem is that speed of tactical execution can make up for a lot of strategic thought. The famous example: you can rush a line of siege tanks with zerglings if you can micro them fast enough[0]. [0]: https://www.youtube.com/watch?v=IKVFZ28ybQs

I hope that in the interest of fair play they'll limit their AI to 300 APM or so. Make it win not on mechanical execution, but on decision making.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#479

Post by darkmighty »

soup10 wrote: post #428
Florin_Andrei wrote: post #418

Earlier quoted context omitted.

FWIW, before AlphaGo defeated Fan Hui 2-dan last year, everyone was saying that would not be possible before 2025 or so. That was the consensus.

People that try to predict the future in ai are breathing hot air more often then not.

Serious predictions actually inferred from the progress of existing (MCTS and others before) bots, which was something like 1 stone every two years (I don't recall the details, but it's easy to find out there). Top professionals were estimated to be something like 10 stones stronger than the best bot at 2008, so 2025 wouldn't sound too conservative.

"At the US Congress 2008, he [Myungwan Kim] also played a historic demonstration game against MoGo running on an 800 processor supercomputer. With a 9 stone handicap, MoGo won by 1.5 points. At the 2009 congress, he played another demonstration game against Many Faces of Go running on 32 processors. Giving 7 stones handicap, Kim won convincingly by resignation."

(Kim Myung Wan (born 1978) is a 9d Korean professional who has taken up residence in the Los Angeles area as of 2008)

More information here, with a nice graph:

http://senseis.xmp.net/?ComputerGo

http://i.imgur.com/RvQsf6v.png

You can see progress seemed to be slow at 2012.

Re: AlphaGo beats the world champion Lee Sedol in first of five matches

#480

Post by andreyk »

terryf wrote: post #414
GolDDranks wrote: post #351

Earlier quoted context omitted.

This is basically a combination. A "traditional" chess program would use a tree search, but trees get quickly ot of hand since they grow exponentially. The trick is to prune them, and they trained a network to do that. It selects just the moves that look good to it. (It has some level of randomness to it, too) After reaching deep enough in the search tree, they use another network to evaluate who's winning. Usually t…

right, yes, but my question was meant to be a bit more general - this and various other results have shown that it is possible to train a deep net to do a specific task very successfully - my question was if it's possible to train it to do two or more tasks as successfully or will the network then have to be exponentially larger. I suppose there is no known way to "combine" trained networks together.

The standard is to have each neural net be trained just for one task, as you say; there may be research into multi-skill neural nets but I have yet to see any. AlphaGo in particular is extremely specialized to Go, even in terms of how the algorithm is implemented.
Post reply on HN