Live data from Hacker News

What Google DeepMind Means for A.I.

newyorker.com

31–40 of 94 posts

Re: What Google DeepMind Means for A.I.

#31
This is impressive. The current approach will work only for games where the whole state is on-screen and planning isn't required. A pure reactive system will work for that.

I used to say that a key component of AI that was missing was the ability to get through the next few seconds of life without falling down or bumping into anything. I went through Stanford CS when the top-down logicians were in charge of AI. That approach was totally incapable of dealing with the real world. Now we're seeing the systems needed to deal with the real world in the short term starting to work.

Once you can deal with the next few seconds, a strategy module can be added to give goals to the low level system. This is very clear in the video game context. As the game playing programs advance beyond the 2D full-screen games, they'll need a low-level system to handle the next moves ("don't fall off platform", "jump to next platform", "shoot at target" are primitives for the 2D sidescroller era) and some level of planner to handle tactical matters and strategy.

It's possible to explicitly build hierarchical systems like that now, using classical planning techniques to modify the goals of a machine learning system. It's not yet possible to get a hierarchical system to emerge from machine learning. Medium term planning as an emergent behavior is a near term big challenge for AI.

Beyond such a two-level system, we're going to need intercommunicating components that do different parts of the problem. The components may be evolved, while the architecture may be designed. When AI systems can design such architectures, they're probably ready to take over.

Re: What Google DeepMind Means for A.I.

#32

Another issue that this article sort of touches on but doesn't make explicit: the real world is not a Markov decision process. There are complex, variable-order time dependencies which we are barely aware of but which influence our thinking every second of every day. Trying to model this in software leads to an exponential increase storage and time complexity. The curse of dimensionality has been with us ever since B…

Recurrent neural networks with e.g. Long Short Term Memory (https://en.wikipedia.org/wiki/Long_short_term_memory) can model very long term dependencies effectively, and there has been some work lately that get good results with simpler models.

They don't use it because it's computationally expensive and totally unnecessary for Atari games, but it's certainly possible.

Re: What Google DeepMind Means for A.I.

#33
post #9

Earlier quoted context omitted.

There are two sides to this, world simulation and AI. As the other replies already said, current AI isn't close to toddler-level (there's no reasoning going on in the DeepMind work, just statistical correlation). We're also way off on the world simulation side - show me a realistic world simulator that can run close to realtime. Physically-based rendering is indeed impressive but this only accounts for visual percept…

> (there's no reasoning going on in the DeepMind work, just statistical correlation) I've seen 100 people make this statement and mean 100 different things, so I just wanted to clarify: How are you defining "reasoning" here as distinct from statistical correlation?

Reasoning involves inferring and applying causation which is different from correlation [1]. One can possibly define process of "understanding" as building a "model" of the system where previously unseen events can be predicated or justified using the model. The big difference in "human understanding" seems to be that we can extract fairly minimal set of laws that govern the system from our observations that we can communicate and apply very efficiently.

1. http://en.wikipedia.org/wiki/Correlation_does_not_imply_caus...

Re: What Google DeepMind Means for A.I.

#34
post #31

This is impressive. The current approach will work only for games where the whole state is on-screen and planning isn't required. A pure reactive system will work for that. I used to say that a key component of AI that was missing was the ability to get through the next few seconds of life without falling down or bumping into anything. I went through Stanford CS when the top-down logicians were in charge of AI. That…

I remember an AI researcher (I forget who) recently said something to the effect that early AI research produced all sorts of planning algorithms. E.g. the top down camp of AI. But they weren't capable of working with the real world because we didn't have very good low level perception. E.g. this complicated algorithm for planning the robot's actions, but it depended on getting input about where objects are.

Now we have decent low level perception from the bottom-up camp of AI, but they are limited by a lack of high level stuff like planning and reasoning.

But you are right that there is no obvious way to just combine these wildly different algorithms without lots of human guidance.

Re: What Google DeepMind Means for A.I.

#35
post #33

Earlier quoted context omitted.

> (there's no reasoning going on in the DeepMind work, just statistical correlation) I've seen 100 people make this statement and mean 100 different things, so I just wanted to clarify: How are you defining "reasoning" here as distinct from statistical correlation?

Reasoning involves inferring and applying causation which is different from correlation [1]. One can possibly define process of "understanding" as building a "model" of the system where previously unseen events can be predicated or justified using the model. The big difference in "human understanding" seems to be that we can extract fairly minimal set of laws that govern the system from our observations that we can c…

That's not really relevant. These models can easily infer causation by seeing what happens as a result of their actions.

Re: What Google DeepMind Means for A.I.

#36

"also discovered a way to win [breakout] that its creator never imagined" I don't understand. We often would bounce balls between the top wall and the bricks while playing breakout on our Atari 2600 back in the day. And I wouldn't say we were all that good (it didn't happen right away).

Yeah. It's a bit odd that the author didn't know this or discuss his thoughts with somebody who knew this (which would be most people who've played Breakout, it often happens accidentally).

edit: Having just watched the source video, she may be actually referring to the creator of the AI and just badly rephrasing what the guy in the video says ( he says that they didn't expect the AI to be able to work that out with the abilities they had given it ).

Re: What Google DeepMind Means for A.I.

#37

"video games" (read "world simulator"). The important thing about their work is that it is deliberately marching down the path of more and more complex world simulations. We experience the world at one second per second. To learn to walk we must first fall, and we fall at 32 feet/second^2. There's a hard limit on how fast we can make mistakes (like tripping) and so there is a hard limit on how fast we can learn. Comp…

>We experience the world at one second per second

There is evidence from animal studies that the hippocampus (a brain structure critical for memory) can 'replay' remembered events at 10-20x speedup. See, for example: http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ Video at: http://youtu.be/Bv7zN2Or6Mg

(Full-disclosure: I am the first author.)

And in fact the OP uses biologically-inspired off-line replay as part of their learning algorithm.

Re: What Google DeepMind Means for A.I.

#39

I really like this line of work and I expect will grow quite substantially over the next few years. Of course, Reinforcement Learning has been around for a long time. Similarly, Q Learning (the core model in this paper) has been around a very long time. What is new is that normally you see these models applied to toy MDP problems with simple dynamics, and linear Q function approximations for fear of non-convergence e…

Thanks, karpathy!

I did submit your JS implementation to HN when I came across it: https://news.ycombinator.com/item?id=9108738

Monte Carlo Tree Search could be the missing link. In other words, use DQN to model the world and map actions to a value function. Then use playouts and backpropogation of action tree results to find tactics. Of course, it does not solve the big question: how to model "memories" and "inferences"? Indeed, very exciting times for AI/ML!

Re: What Google DeepMind Means for A.I.

#40

> “They can find their way across a room,” Mason said. “They can see stuff, and as the light and shadows change they can recognize that it’s still the same stuff. They can understand and manipulate objects in space.” Isn't this just adding extra dimensions to the input space? We have 2D now (plus time?), we're missing Z, sound, sensation, maybe emotions. Each added dimension gives the algorithm exponentially more bit…

I think the distinction is that the algorithm gets to observe the 2D space directly, whereas the whole notion of 3D space has to be learned from 2D projections. Also, note that the games they do best on have a nice clear objective function: Montezuma's Revenge on the other hand does not have such a numeric objective to optimise. To be fair, Hassabis does freely concede these limitations in his talks (at least in the…

That's a good point about Montezuma's Revenge, and similar games. Very rarely does a greedy algorithm help you in that game unlike in a breakout style game where you simply want to acquire as many points as quickly as possible.
Post reply on HN