Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that imagine and plan

deepmind.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Agents that imagine and plan

#51

Post by deepnet »

> particularly in programs like AlphaGo, which use an ‘internal model’ to analyse how actions lead to future outcomes in order to to reason and plan.

I was under the impression that AlphaGo makes no plan but responds to the current board state with expert move probabilites that prunes MCTS random playouts.

There is no plan (AFAIK) or strategy in the AlphaGo papers so I find this statement that AlphaGo is an imaginative planner quite curious.

Perhaps someone can reconcile these statements or correct my knowledge of AlphaGo ?

Very interesting papers, it will be nice to see the imagination encoder methods applied to highly stochastic enviroments or indeed a robot in the real world.

Re: Agents that imagine and plan

#52

Post by bambax »

interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

Of course. This is extremely annoying, esp. now that the Internets are choke-full of counter examples. Descartes wrote many stupid things about animals being "automatons", but at least he had the excuse of living in a pre-Youtube erra.

> When placing a glass on the edge of a table, for example, we will likely pause to consider how stable it is and whether it might fall. On the basis of that imagined consequence we might readjust the glass to prevent it from falling and breaking.

Or, if you're a cat, you might push it over the edge for the fun of it:

https://www.youtube.com/watch?v=RI1rv3re7as

In fact, the cat in this video appears to have more imagination than the paper's authors.

Re: Agents that imagine and plan

#53

Post by interfixus »

zimpenfish wrote: post #49
interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

> I can think of no other explanation Well, just because you can't think of one, doesn't mean your explanation is correct, surely. This could easily be explained by an instinctual "hide food remnants to avoid attracting bigger things".

In some formal scheme yes. In the actual situation no, it could not easily. Or we can reduce the question to a squabble of semantics: Alright, the dog's actions were not conscious and actively planned, but then neither are ours. I fail to see the fundamental difference, and have never really heard a coherent case made that there is one. You are of course right that argument from own lack of imagination is no proof of anything.

Re: Agents that imagine and plan

#54

Post by interfixus »

akvadrako wrote: post #50
interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

> Tell me again about imagination and its distinctly human nature. https://en.wikipedia.org/wiki/Bicameralism_(psychology)

Thank you. Yet another exhibit in the case against psychology as a valid scientific endeavour.

Re: Agents that imagine and plan

#55

Post by bryanrasmussen »

interfixus wrote: post #47

" This form of deliberative reasoning is essentially ‘imagination’, it is a distinctly human ability " A completely unfounded supposition, as so often appears to be the case when some human monopoly is claimed. We didn't magically sprout whole new categories of ability during a measly few million years of evolution. Anecdotally, I see crows getting out out the way of my car. Not confused and haphazardly as many birds…

reminds me of my dog and cat when I was a kid in Germany whenever we left the house the cat, a Siamese who had learned to open doors, would open the door to the garbage and the dog would then pull it into the kitchen to spread on the floor for a party.

Re: Agents that imagine and plan

#56

Post by gone35 »

gradstudent wrote: post #28

I'm not a planning guy but I work in a closely related community so I'm a least somewhat familar with the area. Looking at the first paper ( https://arxiv.org/pdf/1707.06170.pdf ), it seems surprisingly shallow and light on details. So they have a learning system for continuous planning. So what? The AI Planning community has been doing this for ages with MDPs and POMDPs, solving problems where the planning domain ha…

Not my field either but prima facie it does seem suspiciously close to good-old hallucinated feedback-like techniques, POMDPs, etc in the planning / ML-oriented robotics community (see e.g. [1]). Didn't read too carefully though...

[1] Boots, et al. (2011) Closing the learning-planning loop with predictive state representations. http://journals.sagepub.com/doi/10.1177/0278364911404092

Re: Agents that imagine and plan

#57

Post by randallsquared »

interfixus wrote: post #53
zimpenfish wrote: post #49

Earlier quoted context omitted.

> I can think of no other explanation Well, just because you can't think of one, doesn't mean your explanation is correct, surely. This could easily be explained by an instinctual "hide food remnants to avoid attracting bigger things".

In some formal scheme yes. In the actual situation no, it could not easily. Or we can reduce the question to a squabble of semantics: Alright, the dog's actions were not conscious and actively planned, but then neither are ours. I fail to see the fundamental difference, and have never really heard a coherent case made that there is one. You are of course right that argument from own lack of imagination is no proof of…

> Alright, the dog's actions were not conscious and actively planned, but then neither are ours.

Well, some of ours are. At least a few. It's not clear that any of the dog's actions are consciously planned, is it?

Re: Agents that imagine and plan

#58

Post by ebalit »

deepnet wrote: post #51

> particularly in programs like AlphaGo, which use an ‘internal model’ to analyse how actions lead to future outcomes in order to to reason and plan. I was under the impression that AlphaGo makes no plan but responds to the current board state with expert move probabilites that prunes MCTS random playouts. There is no plan (AFAIK) or strategy in the AlphaGo papers so I find this statement that AlphaGo is an imaginati…

In AlphaGo, MCTS is used to explore many plans and select the best. As far as I know, it then execute only the first action of the selected plan, and start a new planning for the next action. As such, it doesn't "stick to the plan", so you could say that it doesn't have a strategy. But the MCTS is definitely a planner.

Re: Agents that imagine and plan

#59

Post by interfixus »

randallsquared wrote: post #57
interfixus wrote: post #53

Earlier quoted context omitted.

In some formal scheme yes. In the actual situation no, it could not easily. Or we can reduce the question to a squabble of semantics: Alright, the dog's actions were not conscious and actively planned, but then neither are ours. I fail to see the fundamental difference, and have never really heard a coherent case made that there is one. You are of course right that argument from own lack of imagination is no proof of…

> Alright, the dog's actions were not conscious and actively planned, but then neither are ours. Well, some of ours are. At least a few. It's not clear that any of the dog's actions are consciously planned, is it?

I should think it fairly clear, unless you propose some definite, qualitative difference between the dog and the rest of us. It's not clear that such a difference exists, is it?

Re: Agents that imagine and plan

#60

Post by deepnet »

ebalit wrote: post #58
deepnet wrote: post #51

> particularly in programs like AlphaGo, which use an ‘internal model’ to analyse how actions lead to future outcomes in order to to reason and plan. I was under the impression that AlphaGo makes no plan but responds to the current board state with expert move probabilites that prunes MCTS random playouts. There is no plan (AFAIK) or strategy in the AlphaGo papers so I find this statement that AlphaGo is an imaginati…

In AlphaGo, MCTS is used to explore many plans and select the best. As far as I know, it then execute only the first action of the selected plan, and start a new planning for the next action. As such, it doesn't "stick to the plan", so you could say that it doesn't have a strategy. But the MCTS is definitely a planner.

Yes absolutely, I think your explication is perfectly correct.

Though (IMHO) MCTS is better characterised as evaluating moves rather than exploring plans.

The MCTS only explores the moves in order of likelyhood using the most basic of heuristics, random playout.

The Net outputs likely moves based only the current board position, it formulates no strategy.

No state is stored across moves - each play is independent, relying only on the current board position.

I still don't see anything anywhere in AlphaGo that is a plan, trajectory or strategy.

Neither is there an evaluation of the opponent nor any attempt to outwit them.

That it performs so astonishingly well without a plan is very very interesting and should perhaps give us pause - is planning a hubris ? Do we undervalue our use of heuristics in our own behaviour ?

Post reply on HN