Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

241–250 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#241

Post by red75prime »

sobellian wrote: post #193

The catch is that this isn't quite zero human knowledge, since the tree search algorithm is a human discovery, and not one that came easily to humans. It also massively cuts down on the search space for an appropriate policy function. That means that this setup isn't necessarily general. How applicable is MCTS to games with asymmetric information, a la Starcraft? What about games that can't quite be modeled with an a…

> What about games that can't quite be modeled with an alternating turn-based game tree like bughouse?

Train a network which predicts future state of the game, given current state and input. Train a network which generates sensible inputs, given current state. Use MCTS.

Re: AlphaGo Zero: Learning from scratch

#244

Post by Tenoke »

YeGoblynQueenne wrote: post #230
ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

I spent an hour of my life that I'll never get back reading Yudkowski's overly-long article and I believe I can summarise it thusly: "We don't know how AGI will arise; we don't know when; we don't know why; we don't know anything at all about it and we won't know anything about it until it's too late to do anything anyway; We must act now!!" The question is- if we don't know anything about this unknowable threat, how…

>The question is- if we don't know anything about this unknowable threat, how can we protect ourselves against it? In fact, since we're starting from 0 information, anything we do has equal chances of backfiring and bringing forth AGI as it has of actually preventing it. Yudkowski is calling for random action, without direction and without reason.

Are you sure you read the essay? That's literally the question he answers.

At any rate, we do have more than '0 information', and if you make an honest effort to think of what to do you can likely come up with better than 'random actions' for helping (as many have).

Re: AlphaGo Zero: Learning from scratch

#245

Post by sobellian »

red75prime wrote: post #241
sobellian wrote: post #193

The catch is that this isn't quite zero human knowledge, since the tree search algorithm is a human discovery, and not one that came easily to humans. It also massively cuts down on the search space for an appropriate policy function. That means that this setup isn't necessarily general. How applicable is MCTS to games with asymmetric information, a la Starcraft? What about games that can't quite be modeled with an a…

> What about games that can't quite be modeled with an alternating turn-based game tree like bughouse? Train a network which predicts future state of the game, given current state and input. Train a network which generates sensible inputs, given current state. Use MCTS.

Bughouse, starcraft, and other important games need to be modeled as simultaneous-decision games. Plain-vanilla MCTS is designed for alternating-decision games.

To see why this is important, consider why min-max (which MCTS approximates) actually works. At any given point, the equilibrium strategy for the player to move is the move that maximizes their payoff, and the utility for each move can be found recursively.

In simultaneous decision games, calculating the equilibrium strategy (which may even be a mixed strategy) is more complicated. See http://mlanctot.info/files/papers/cig14-smmctsggp.pdf for various ways in which MCTS can be extended to simultaneous-decision games.

It'll be interesting to see if DeepMind picks up a search algorithm someone else has researched, or if they come up with something entirely new.

Re: AlphaGo Zero: Learning from scratch

#246

Post by igravious »

autarch wrote: post #154
aeleos wrote: post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

> things like Chinese starts that are specific to certain cultures While it's true that there are national styles of play, the Chinese opening is not called that because it's really popular among Chinese people. It's called that because a particular Chinese pro helped popularize it, even though it was invented by a Japanese amateur. See https://en.wikipedia.org/wiki/Chinese_opening for some more info. FWIW, I (a cauc…

> talk about things like Chinese starts that are specific to certain cultures

Came here to make this point.

It's Chinese Opening, not Chinese start – similarly recall that you have the French Defense / Italian Defense / Scandinavian Defense among chess opening variations and none of these implies that that opening variation is specific to that culture or nation.

Re: AlphaGo Zero: Learning from scratch

#247

Post by mrec »

AlexCoventry wrote: post #51
panic wrote: post #40

Earlier quoted context omitted.

Tree search is also used during play. In the paper, they pit the pure neural net against other versions of the algorithm -- it ends up slightly worse than the version that played Fan Hui, at about 3000 ELO.

Oh, so it's just not using rollouts to estimate the board position? Thanks for the clarification.

It doesn't use rollouts at all:

> AlphaGo Zero does not use “rollouts” - fast, random games used by other Go programs to predict which player will win from the current board position. Instead, it relies on its high quality neural networks to evaluate positions.

Re: AlphaGo Zero: Learning from scratch

#248

Post by red75prime »

sobellian wrote: post #245
red75prime wrote: post #241

Earlier quoted context omitted.

> What about games that can't quite be modeled with an alternating turn-based game tree like bughouse? Train a network which predicts future state of the game, given current state and input. Train a network which generates sensible inputs, given current state. Use MCTS.

Bughouse, starcraft, and other important games need to be modeled as simultaneous-decision games. Plain-vanilla MCTS is designed for alternating-decision games. To see why this is important, consider why min-max (which MCTS approximates) actually works. At any given point, the equilibrium strategy for the player to move is the move that maximizes their payoff, and the utility for each move can be found recursively. I…

Thanks for the link. The thing I described roughly corresponds to SUCT.

It's interesting how NN will be able to deal with uncertainty of enemy's state and moves.

Re: AlphaGo Zero: Learning from scratch

#250

Post by derefr »

red75prime wrote: post #238
derefr wrote: post #231

Earlier quoted context omitted.

> We have an existence proof that this should be possible. Not guaranteed. The human brain has diffusion signalling (i.e. neurotransmitters passing out of the synaptic cleft, into a neighbouring one, and activating a receptor on some other spacially-local axon as a result.) And one of those signalling molecules is thought to represent, in its intensity, a confidence-interval bias adjustment (i.e. a pruning bias facto…

That will be a couple of additional terms in activation function. Or am I missing something?

“Neighbouring” is defined in terms of embedding in a metric space and inverse-cube diffusion, rather than anything to do with graphic connectivity.

Also, these signals pile up in the synaptic cleft until they’re picked up, so it’s not just about instantaneous transmissivity as if these were radio signals.

But also also, other stuff like monoamine oxidase is floating about in its own diffusion patterns, cleaning up these signals.

It’s basically like a “scent” communication embodied-actor model, but a very complex one where things like redox reactions with the atmosphere occur.

Oh, and there are “secondary messengers”: signals that trigger other signals that, among other things, inhibit the release of the original signal when received back at the sender, such that an dynamic equilibrium state is reached between the two signal types.

Post reply on HN