Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaGo Zero: Learning from scratch

deepmind.com

Post reply on HN
Print view View original

81–90 of 324 posts

Re: AlphaGo Zero: Learning from scratch

#81

Post by red75prime »

Arcsech wrote: post #20
ericb wrote: post #7

I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…

While this is promising, there's a long way to go between this and the other things you mentioned. Go is very well-defined, has an unequivocal objective scoring system that can be run very quickly, and can be simulated in such a way that the system can go through many, many iterations very quickly. There's no way to train an AI like this for, say, health: We cannot simulate the human body to the level of detail that'…

> most of the problems required for a general AI aren't well-defined.

Do you care to give an example? Are they more or less well defined than find-the-cat-in-the-picture problem?

> Producing a definitive, objective score for a paper clip collection is very difficult if not impossible.

Erm, producing of objective comparison of relative values of Go board positions is still not possible.

Re: AlphaGo Zero: Learning from scratch

#83

Post by hetspookjee »

Amazing results, though I am somewhat frightened by how generic this model is and how it achieved such amazing results. I can't help but think that these same techniques can be used to learn how humans react in certain situaties and how they can be, very subtely, be worked to think in a certain way - one that fits the agenda of whatever party is behind it.

With the mass surveillance that is Google it's quite doable to test for human reactions on certain things. They got the tools to execute a certain plan and evaluate the effectiveness. Ofcourse it can also go in a benelovent way: like what kind of policy will benefit the most people? (semantics of 'benefiting' aside)

I atleast certainly hope these kinds of generic algorithms will be used to generate effective, meaningful policies that truly help the people. Still a far away future but one that gets closer by the day.

Re: AlphaGo Zero: Learning from scratch

#84

Post by [deleted] »

gordon_freeman wrote: post #33
Jyaif wrote: post #4

"It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features." This is amazing! The technology they came up with must be super generic.

I will be interested to see what kind of algorithms they have used to allow AlphaGo to learn from its own moves. Are these pretty generics algos or are these very customized and specific ones that only apply to AlphaGo and the game of Go?

[deleted]

Re: AlphaGo Zero: Learning from scratch

#86

Post by aeleos »

marcosscriven wrote: post #75
aeleos wrote: post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

Possibly a dumb q, but is ‘self play’ in any way related to ‘adversarial’ learning? I don’t see it mentioned in the article, but it reminds me of the principle.

In some ways it is, but the main difference is that adversarial learning (usually) produces a second neural network whose purpose is to exploit weakness is the first. Whereas reinforcement learning does not produce a second neural network to beat the first, it uses what it learned to solely improve the original.

As a side note, the main application I have seen with adversarial learning research is with photo recognition, but I guess you could have an adversarial network exist to help help improve an object recognition network. At that point it would probably become something between adversarial and reinforcement learning. However, with game based reinforcement learning, it doesn't require a second specific network as the adversary, it can easily just be paired against itself.

It isn't a dumb question, they are very similar in some ways. They mainly differ in what exactly the goal of the opponent is. In this case, it is to help improve itself, however in typical adversarial situations it is solely to exploit (become its adversary).

Re: AlphaGo Zero: Learning from scratch

#88

Post by aeleos »

shmageggy wrote: post #72
aeleos wrote: post #52

The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…

The key part from the paper: > To assess the merits of self-play reinforcement learning, compared to learning from human data, we trained a second neural network (using the same architecture) to predict expert moves in the KGS Server data­ set; this achieved state-of-the-art prediction accuracy compared to pre­ vious work 12,30–33 (see Extended Data Tables 1 and 2 for current and previous results, respectively). Supe…

That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence.

EDIT: changed a set of neurons to neural network per andbbergers comments

Re: AlphaGo Zero: Learning from scratch

#89

Post by bradleybuda »

narvind wrote: post #16

How I wish Marvin Minsky would have stayed alive for one more year and seen this. He would have been so happy!

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

“What are you doing?”, asked Minsky.

“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.

“Why is the net wired randomly?”, asked Minsky.

“I do not want it to have any preconceptions of how to play”, Sussman said.

Minsky then shut his eyes.

“Why do you close your eyes?”, Sussman asked his teacher.

“So that the room will be empty.”

At that moment, Sussman was enlightened.

http://www.catb.org/jargon/html/koans.html

Re: AlphaGo Zero: Learning from scratch

#90

Post by SubiculumCode »

hetspookjee wrote: post #83

Amazing results, though I am somewhat frightened by how generic this model is and how it achieved such amazing results. I can't help but think that these same techniques can be used to learn how humans react in certain situaties and how they can be, very subtely, be worked to think in a certain way - one that fits the agenda of whatever party is behind it. With the mass surveillance that is Google it's quite doable t…

I'd only worry if it can outperform humans when there are not rules per se. That is, if I put a queen down in the GO board, and start knocking off stones, moving three times a turn, then take a lighter and burn the go board, the AI responds by decapitating my head.
Post reply on HN