Agents that imagine and plan
deepmind.com
Agents that imagine and plan
1–10 of 62 posts
Re: Agents that imagine and plan
#2https://en.wikipedia.org/wiki/Model_predictive_control
Of course imagining possible outcomes before executing is useful! And it has many uses outside deep learning. No reason to reinvent new words, really. At least without referring to the established ones.
Maybe there is a serious novel idea, but I've missed it.
Basically, if you need to control a complex process (i.e. bring some future outcome in accordance to your plan), you can build a forward model of the system under control (which is simpler than a reverse model), and employ some optimization techniques (combinatorial, i.e. graph-based; numeric derivative-free, i.e. pattern-search; or differential) to find the optimal current action.
Re: Agents that imagine and plan
#3Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such as an "imagination encoder".
Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network? Conversely, if we have to explicitly account for these properties and come up with specific designs to emulate them, how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?
Re: Agents that imagine and plan
#4Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
Imagination doesn't seem learnt to me. Instead learning new concepts adds to the toolbox so to speak.
Re: Agents that imagine and plan
#5Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?
No! There's never been any scientific guarantee that "sufficiently complex" neural networks will give rise to anything in specific as an "emergent property", let alone human cognitive abilities like imagination and planning.
>how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?
Steps to write a deep learning paper (from the Cynic's Guide to Artificial Intelligence):
1) Use a training set orders of magnitude larger than a human could learn from, build a one-trick network that gets superhuman performance on its one trick of a task.
2) Hype it up.
3) Research funding and/or profit and/or world domination!
(World domination has never been supplied when requested.)
Re: Agents that imagine and plan
#6Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
Well, nobody is forced to create models they don't believe in.
But besides some models being useful, as that old adage says, some are also useful-er than others. Adding stuff such as "imagination" functions as a constraint on the number of behavioral patterns that we are willing to consider, and that might lead us to find that one which we're looking for (i.e. "intelligent behavior") faster than a naïve approach.
Besides, it might not be the case that the likelihood of observing "intelligent behavior" increases over the complexity of the behavior generating process.
Re: Agents that imagine and plan
#7Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?
Not necessarily. I think it comes down to what you mean by "sufficiently complex". If we took a classic feedforward Multi-Layer Perceptron and gave it massive amounts of good data, a long time to train, and a nearly unbounded network size, I'm not sure it would ever develop architecture within itself to plan or develop a robust internal model.
Our neurology took millions of years/generations to get where it is today though natural selection. We might want to tip the scales a bit by engineering the broad architectural pieces and letting emergent behavior fill the gaps.
Although it would be fun to try producing human level intelligence by seeding a physics simulation of primordial soup and letting it run for millions of "years", I don't think that's feasible for most researchers.
Re: Agents that imagine and plan
#8Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
The human brain has specialized structures in it, it isn't a homogeneous mass from which all parts of human cognition emerge once you have enough brain cells (see elephant brain size vs. human brain size). If you've ever seen anything else designed by evolution, you'll know it generally tends to be a grab-bag of weird tricks all combined together in a way that somehow works. We don't know what all the tricks are, nor which are necessary or sufficient to create human-like intelligence.
There are also a lot of indications that ultimately you need some tricks (i.e. specialized portions of the architecture that bias the kinds of solutions the AI can learn) to be able to learn effectively in the environments we're interested in. For example, we know that there is a time dimension to agent tasks, and that objects don't pop in and out of existence, they tend to exist continuously. These are biases we are free to add to a learning system without worrying about it limiting the ultimate intelligence of the system.
In the limit, the No Free Lunch theorems indicate that there's no such thing as a general learning system that doesn't sacrifice performance on some kinds of tasks. The goal of AI research is to sacrifice performance on tasks that we'll never encounter in favor of getting good performance on tasks we care about.
Re: Agents that imagine and plan
#9Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
The human brain has specialized structures in it, it isn't a homogeneous mass from which all parts of human cognition emerge once you have enough brain cells (see elephant brain size vs. human brain size). If you've ever seen anything else designed by evolution, you'll know it generally tends to be a grab-bag of weird tricks all combined together in a way that somehow works. We don't know what all the tricks are, nor…
> If you've ever seen anything else designed by evolution, you'll know it generally tends to be a grab-bag of weird tricks all combined together in a way that somehow works. We don't know what all the tricks are, nor which are necessary or sufficient to create human-like intelligence.
That is precisely the core of my interrogation. The papers mentioned in the article seem to be about "hand designing" the weird tricks; shouldn't the goal be to build a system that enables the emergence of these weird tricks without involving human design?
Re: Agents that imagine and plan
#10https://en.wikipedia.org/wiki/Model_predictive_control Of course imagining possible outcomes before executing is useful! And it has many uses outside deep learning. No reason to reinvent new words, really. At least without referring to the established ones. Maybe there is a serious novel idea, but I've missed it. Basically, if you need to control a complex process (i.e. bring some future outcome in accordance to your…
The link between imagining and deep learning is rather in the opposite direction - it has always been obvious that imagining possible outcomes before executing would be useful, but the novelty is that deep learning has allowed them to actually make "imagination" that works.
MPC is an useful concept if you have a predictive model that's at least vaguely close to the actual behavior. In some contexts (e.g. modeling of particular industrial systems) programmers could build such a model, but in the general case that's absolutely not feasible, the world is full with problems where, practically speaking, you can not manually build a forward model of the system under control.
So this article is about initial research on systems that can construct such a predictive model/imagination from experience, with a proof of concept that the current deep learning approaches allow us to build systems that can learn such predictive models (which wasn't really possible before) and further development of this concept seems to be the way how we can actually apply things like MPC to problems where we won't build a forward model ourselves; and in the long run, that means pretty much all problems.