Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpha Go Zero: How and Why It Works

tim.hibal.org

Post reply on HN
Print view View original

11–20 of 116 posts

Re: Alpha Go Zero: How and Why It Works

#11

Post by kbenson »

ben_w wrote: post #8
javajosh wrote: post #3

"On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_) Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-repres…

If it is optimising all your decisions, even the mundane ones, is it really your life any more? If it plugged into your brainstem and took over, by definition nobody else would notice the difference (you always did what it said anyway, or you’d be suboptimal), so why not just flood the 20 watt carbon-based neural network with bliss drugs and let the silicon-based neural network be the person instead?

You set the destination, it sets the route. Maybe you tell it you feel like you aren't getting enough exercise, or you need more time at night for a friend. It schedules the mundanity around those goals. I would love that.

Re: Alpha Go Zero: How and Why It Works

#12

Post by GolDDranks »

QML wrote: post #6

What makes this different from a minimax algorithm with alpha-beta pruning?

Minimax algorithm with alpha-beta pruning is an exhaustive search algorithm. MCTS is a probabilistic algorithm that optimises for converging faster for a partial "good enough" result.

Introducing MCTS for Go playing programs in 2006 brought upon a revolution in playing strength. For the first time Go programs stood a chance against a dan-level amateur.

Re: Alpha Go Zero: How and Why It Works

#14

Post by Mageek »

QML wrote: post #6

What makes this different from a minimax algorithm with alpha-beta pruning?

The branching factor for games like Go is too high. Alpha Go Zero uses a stochastic policy to focus its attention on useful branches. This policy is optimized alongside the predicted value for each node.

Re: Alpha Go Zero: How and Why It Works

#17

Post by zone411 »

danielvf wrote: post #4
EGreg wrote: post #2

I wonder how the STYLE of Alpha Go Zero is regarded by human experts. Is it far different from AlphaGo? Why bother learning from AlphaGo if they can learn from AlphaGo Zero? Did they unleash a second "Master" program? I am wondering if the "better" strategy moves are now super wacky and weird and break all theory.

At least initial reports are that alphaGo Zero is more human-like than Master. Zero packs even more of the inhuman ability to pick the most critical part of the board for each move, but less weird looking stuff. In fact, one of the obvious differences between AlphaGo Zero and top human players, is much more play on safe opening spots, which has been out of fashion among human pros for a hundred years or so.

Interesting. It may be possible that while the style of play of Master is able to exploit the errors of human players easily and score convincing victories by many stones, Zero is closer to playing Go perfectly, even if it means winning by one stone. There is a chance that Zero would not score such convincing victories over humans as Master but is much less exploitable. If a strong player plays obvious novices (and humans are now at a level of novices compared to these programs), it doesn't have to worry about taking more unsound risks.

Re: Alpha Go Zero: How and Why It Works

#18

Post by comex »

yters wrote: post #9

Are there adversarial examples for Alpha Go Zero?

It seems like it would be drastically harder to attack a Go-playing network compared to an image classifier, just because there’s so much less freedom in input. With an image, you can vary each pixel independently however you want. In a game of Go, on any given turn there are probably only a handful of moves that don’t amount to throwing away territory, which would probably be harmful enough to offset any gain by ‘confusing’ the AI. Not only that. the list of suitable moves depends on the AI’s choices in addition to its opponent’s, making the state space highly discontinuous. Who knows, though… maybe there’s some vulnerability so severe that you can spend half the game wasting your moves and win anyway, because the AI gets so confused that it stops playing with even basic competence. I guess that image classifiers giving 99.9% confidence to the wrong class also seems pretty ‘incompetent’… and it would be very cool to see.

Another obstacle is that whereas an image classifier is basically a purely learned function, Monte Carlo tree search has a hard-coded element of applying potential moves to find new states to explore - always following the game’s rules correctly. If you go more than a few moves ahead, it has a relatively unpredictable choice of what states to explore; but as you get closer to the current move, I think it starts to look a bit more like brute force. So if you want the AI to make a really bad move - one that has severe consequences in the next few turns - you need to not just make the NN think it looks good, but that the game state still looks good even after you make any possible counterplay.

Disclaimer: I don’t actually know much about either neural networks or Go.

Re: Alpha Go Zero: How and Why It Works

#19

Post by baudehlo »

ben_w wrote: post #8
javajosh wrote: post #3

"On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_) Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-repres…

If it is optimising all your decisions, even the mundane ones, is it really your life any more? If it plugged into your brainstem and took over, by definition nobody else would notice the difference (you always did what it said anyway, or you’d be suboptimal), so why not just flood the 20 watt carbon-based neural network with bliss drugs and let the silicon-based neural network be the person instead?

Sort of like Surrogates: http://m.imdb.com/title/tt0986263/

Re: Alpha Go Zero: How and Why It Works

#20

Post by gwern »

yters wrote: post #9

Are there adversarial examples for Alpha Go Zero?

Possibly but no one has demonstrated them, and there are probably many fewer in Zero than in previous AlphaGos. Previously AlphaGos suffered from problems which Silver calls 'delusions' (https://www.reddit.com/r/MachineLearning/comments/76xjb5/ama...) ie persistent long-term misevaluations of specific board positions, which were however (entirely? how would we know if not?) fixed by Zero's self-play - arguably these are not 'adversarial examples' in the sense you're thinking of but then I'm not sure what would constitute an adversarial example for a Go agent, so maybe the AlphaGo 'delusions' were the equivalent.

(On a side note, the more I think about Anthony et al 2017/AlphaGo Zero's tree/expert iteration, the more beautifully simple it appears to me.)

Post reply on HN