Evaluating different outcomes far ahead may be very computational intensive. One thing that AlphaGO shows is that a simple approach with Monte Carlo tree search can drastically cut down the search space. The "imagine" part could be just guided random walk ahead in planning, with something like Monte Carlo tree search.
Agents that imagine and plan
21–30 of 62 posts
Re: Agents that imagine and plan
#22I'm likely completely missing the point but how is this concept of imagination different from looking ahead in a search tree? Isn't exploring a search tree like in Chess or Go exploring future possibilities and their consequences before you decide on what to do next?
Re: Agents that imagine and plan
#23Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such…
>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?
Why would you think that? We have no theoretical knowledge of how human "consciousness" emerges, and obviously no experimental data either.
>Conversely, if we have to explicitly account for these properties and come up with specific designs to emulate them, how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?
We don't know how far the path leads, but the capabilities of the past 5-ish years progress are leaps and bounds beyond what anything else can do, and this is part of the work of pushing further down that path.
Re: Agents that imagine and plan
#24https://en.wikipedia.org/wiki/Model_predictive_control Of course imagining possible outcomes before executing is useful! And it has many uses outside deep learning. No reason to reinvent new words, really. At least without referring to the established ones. Maybe there is a serious novel idea, but I've missed it. Basically, if you need to control a complex process (i.e. bring some future outcome in accordance to your…
Precisely what I wanted to say. The 'imagination' they describe is simply reasoning about the future based on current information (with implied uncertainty). Large chunks of any AI textbook are about analyzing future states from present states, planning actions to manipulate that, and the effects of uncertainty on it.
'Imagination' isn't even a good word for it---in conversational English, we often use the word for thinking about models of fictitious states that can't happen, which has subtle value for humans, but not yet for machines.
Re: Agents that imagine and plan
#25The obvious caveat: this is quite far away from my field of expertise. Doubly so, because I'm not an expert in neural net ML and neither in cognitive science. So take this with spoonful of salt. But anyhow , I don't like the word "imagine" here. It seems suggest cognitive capabilities that their model probably does not have. As far as I do understand the papers, their model builds (in unsupervised fashion which sound…
Can you elaborate on what qualitative difference do you see between imagination-as-you-understand-it and an internal simulation of a nonexistent (maybe future, maybe never happening) state of an agent's environment or inputs? There's an obvious quantitative difference - their environment is much simpler than ours, and their "imagination" is bound to imagining the near future (unlike us), but conceptually, where do yo…
Yeah. I doubt that their machine can "imagine no possessions", or that it would have much utility even if it could.
Re: Agents that imagine and plan
#26Off topic: I posted this exact article four days ago: https://news.ycombinator.com/item?id=14813807
In the past, when I post exact duplicates, HN redirects me and automatically upvotes the original instead. I wonder why this doesn't always happen. (I'm not bothered, just curious.)
Double off topic: It's very interesting to see how much difference timing makes. My original had a single upvote, and this hit the front page.
Re: Agents that imagine and plan
#27I'm confused. I thought AI was already about this.
Are they introducing something new, or is it just gimmick and buzzwords?
Re: Agents that imagine and plan
#28I'm not a planning guy but I work in a closely related community so I'm a least somewhat familar with the area.
Looking at the first paper (https://arxiv.org/pdf/1707.06170.pdf), it seems surprisingly shallow and light on details. So they have a learning system for continuous planning. So what? The AI Planning community has been doing this for ages with MDPs and POMDPs, solving problems where the planning domain has some discrete variables and some continuous variables. Here's a summary tutorial from Scott Sanner at ICAPS 2012: http://icaps12.icaps-conference.org/planningschool/slides-Sa...
Speaking of ICAPS: this conference is the primary venue for disseminating scientific results to researchers in the area. Yet the authors here cite exactly one ICAPS paper. WTF?
My bullshit detector is blaring.
Re: Agents that imagine and plan
#29Earlier quoted context omitted.
Can you elaborate on what qualitative difference do you see between imagination-as-you-understand-it and an internal simulation of a nonexistent (maybe future, maybe never happening) state of an agent's environment or inputs? There's an obvious quantitative difference - their environment is much simpler than ours, and their "imagination" is bound to imagining the near future (unlike us), but conceptually, where do yo…
Yeah. I doubt that their machine can "imagine no possessions", or that it would have much utility even if it could.
You're right, but I think this paper is the first step on a (potentially very, very) very long road to building machines that could "imagine no possessions"
Re: Agents that imagine and plan
#30https://en.wikipedia.org/wiki/Model_predictive_control Of course imagining possible outcomes before executing is useful! And it has many uses outside deep learning. No reason to reinvent new words, really. At least without referring to the established ones. Maybe there is a serious novel idea, but I've missed it. Basically, if you need to control a complex process (i.e. bring some future outcome in accordance to your…
Moreover, data-driven control isn't a new concept. It's not my field so I can't comment on what's new here, but I've heard about learning dynamics and rewards in a control theory context plenty of times.