Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that imagine and plan

deepmind.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Agents that imagine and plan

#41

Post by habitue »

tnecniv wrote: post #30
ansgri wrote: post #2

https://en.wikipedia.org/wiki/Model_predictive_control Of course imagining possible outcomes before executing is useful! And it has many uses outside deep learning. No reason to reinvent new words, really. At least without referring to the established ones. Maybe there is a serious novel idea, but I've missed it. Basically, if you need to control a complex process (i.e. bring some future outcome in accordance to your…

Moreover, data-driven control isn't a new concept. It's not my field so I can't comment on what's new here, but I've heard about learning dynamics and rewards in a control theory context plenty of times.

In a control theory context, we're likely talking about inferring a small handful of parameters where the relationships between them are well known. In this paper they're inferring the entire dynamics of an environment from thousands of raw pixel values. This is not something that admits a tractable exact optimization

Re: Agents that imagine and plan

#42

Post by red75prime »

seanwilson wrote: post #37
sullyj3 wrote: post #32

Earlier quoted context omitted.

A search tree in something like chess is quite small, and very discrete. You can enumerate every possible action, and exploring the tree to a useful depth is computationally tractable. By contrast, for an agent operating in a complex environment, like a robot in the real world, even if you somehow came up with a coherent process for listing every possible action the robot could take, you might not even be able to sto…

> Moreover, the real world is (for practical purposes) continuous. The robot would have the option of engaging one of it's motor for one millisecond, or two milliseconds, or three milliseconds, etc. Are there not similar techniques to search trees that are used here? Obviously you wouldn't enumerate all options but you'd think you could guess at some practical ones then guess options between the most promising. Eithe…

Ability to generate candidate paths in continuous search space is a bit more general than pruning prebuilt search tree.

Search tree is an approximation of continuous search problems and needs to be built by someone. This approach builds its own search tree.

Re: Agents that imagine and plan

#43

Post by boulos »

jtraffic wrote: post #26

Off topic: I posted this exact article four days ago: https://news.ycombinator.com/item?id=14813807 In the past, when I post exact duplicates, HN redirects me and automatically upvotes the original instead. I wonder why this doesn't always happen. (I'm not bothered, just curious.) Double off topic: It's very interesting to see how much difference timing makes. My original had a single upvote, and this hit the front p…

The merging is fairly narrowly windowed in time (I think ~hours not >1 day). Sometimes the mods will send you an email (if you have one stored in your account profile) and ask you to repost with a front-page bonus attached. But yeah, timing is everything :).

Re: Agents that imagine and plan

#44

Post by tnecniv »

habitue wrote: post #41
tnecniv wrote: post #30

Earlier quoted context omitted.

Moreover, data-driven control isn't a new concept. It's not my field so I can't comment on what's new here, but I've heard about learning dynamics and rewards in a control theory context plenty of times.

In a control theory context, we're likely talking about inferring a small handful of parameters where the relationships between them are well known. In this paper they're inferring the entire dynamics of an environment from thousands of raw pixel values. This is not something that admits a tractable exact optimization

That may be fair. I only read the first paper with the spaceships and mazes, which are much more traditional problems.

It sounds like the second paper is the more interesting one from your description though, so I will give that a read.

Re: Agents that imagine and plan

#45

Post by Faith_ »

habitue wrote: post #14
PeterisP wrote: post #10

Earlier quoted context omitted.

The link between imagining and deep learning is rather in the opposite direction - it has always been obvious that imagining possible outcomes before executing would be useful, but the novelty is that deep learning has allowed them to actually make "imagination" that works. MPC is an useful concept if you have a predictive model that's at least vaguely close to the actual behavior. In some contexts (e.g. modeling of…

> systems that can construct such a predictive model/imagination from experience I just want to emphasize this point as the crux here. We have many many techniques for AI that involve doing roll-outs once a smart human with domain knowledge hands the system a fully-formed model of the dynamics. Not so many where the dynamics are learned

I've made $64,000 so far this year working online and I'm a full time student. Im using an online business opportunity I heard about and I've made such great money. It's really user friendly and I'm just so happy that I found out about it. Heres what I do, ===http://www.millionaireprofit.cf/

Re: Agents that imagine and plan

#46

Post by red75prime »

red75prime wrote: post #42
seanwilson wrote: post #37

Earlier quoted context omitted.

> Moreover, the real world is (for practical purposes) continuous. The robot would have the option of engaging one of it's motor for one millisecond, or two milliseconds, or three milliseconds, etc. Are there not similar techniques to search trees that are used here? Obviously you wouldn't enumerate all options but you'd think you could guess at some practical ones then guess options between the most promising. Eithe…

Ability to generate candidate paths in continuous search space is a bit more general than pruning prebuilt search tree. Search tree is an approximation of continuous search problems and needs to be built by someone. This approach builds its own search tree.

It also paves the way to algorithms which can solve problems by repurposing available actions for achieving unintended goals, and by creating new high-level actions from low-level ones.

That is solving problems creatively.

Re: Agents that imagine and plan

#47

Post by interfixus »

"This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability"

A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution.

Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds do, but in calculated, deliberate, unhurried steps to somewhere just outside my trajectory - steps which clearly takes into account such elements as my speed and the state of other traffic on the road. Furthermore, when it's season for walnuts and the like, they'll calmly drop their haul on the asphalt, expecting my tyres to crush it for them. This - in my rural bit of Northern Europe - appears to be a recent import or invention; I never saw it done until two years ago.

And there's The Case of the Dog and the Peanut Butter Jars. My dog, my peanut butter jars, and they were empty, but not cleaned. Alone at home, she found them one day, and clearly had experimented on the first one, which had bitemarks aplenty on the lid. The rest she managed to unscrew without damage. Having licked the jars clean, apparently she got to thinking of the grumpy guy who woul eventually be coming home. I can think of no other explanation why I found the entire stash of licked-clean jars hidden - although not succesfully - under a rug.

Tell me again about imagination and its distinctly human nature.

Re: Agents that imagine and plan

#48

Post by PeterisP »

seanwilson wrote: post #22

I'm likely completely missing the point but how is this concept of imagination different from looking ahead in a search tree? Isn't exploring a search tree like in Chess or Go exploring future possibilities and their consequences before you decide on what to do next?

The difference is that in chess or go, generating a search tree is trivial - predicting the world state after a couple turns of go or enumerating all the possible opponent moves in chess takes just a small bit of straightforward code encoding the (simple) rules of the game.

But how does an agent (not you) figure out a search tree of some nontrivial problem? How do you predict what the world state will be after taking some action if a programmer hasn't done that for you? Heck, even how do you predict what the world state might be after a second of doing absolutely nothing in a real-time environment? This is what this research is about.

Re: Agents that imagine and plan

#49

Post by zimpenfish »

interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

> I can think of no other explanation

Well, just because you can't think of one, doesn't mean your explanation is correct, surely. This could easily be explained by an instinctual "hide food remnants to avoid attracting bigger things".

Re: Agents that imagine and plan

#50

Post by akvadrako »

interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

> Tell me again about imagination and its distinctly human nature.

https://en.wikipedia.org/wiki/Bicameralism_(psychology)

Post reply on HN