Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generally capable agents emerge from open-ended play

deepmind.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Generally capable agents emerge from open-ended play

#11

Post by modeless »

xcodevn wrote: post #3

"Analysing the agent’s internal representations, we can say that by taking this approach to reinforcement learning in a vast task space, our agents are aware of the basics of their bodies and the passage of time and that they understand the high-level structure of the games they encounter." Wow, really amazing if true. P.S.: After looking into their paper, it's not that impressive. They use agent's internal states (L…

> it's not that impressive. They use agent's internal states (LSTM cells, attention outputs, etc.) to predict whether it is early in the episode, or whether the agent is holding an object.

That seems like a decent definition of awareness to me. The agent has learned to encode information about time and its body in its internal state, which then influences its decisions. How else would you define awareness? Qualia or something?

Re: Generally capable agents emerge from open-ended play

#12

Post by _f9cu »

xcodevn wrote: post #3

"Analysing the agent’s internal representations, we can say that by taking this approach to reinforcement learning in a vast task space, our agents are aware of the basics of their bodies and the passage of time and that they understand the high-level structure of the games they encounter." Wow, really amazing if true. P.S.: After looking into their paper, it's not that impressive. They use agent's internal states (L…

[dead]

Re: Generally capable agents emerge from open-ended play

#13

Post by leesec »

jcims wrote: post #5
Tarq0n wrote: post #4

Earlier quoted context omitted.

"Aware" is probably overly anthropomorphized language there. What they mean to say is that all these things have become parameterized within the model.

It would be interesting to see what would happen if they added social dynamics between the agents...like some space for theory of mind (what is that agent thinking), mimicry, communication, etc.

From the article: "Because the environment is multiplayer, we can examine the progression of agent behaviours while training on held-out social dilemmas, such as in a game of “chicken”. As training progresses, our agents appear to exhibit more cooperative behaviour when playing with a copy of themselves. Given the nature of the environment, it is difficult to pinpoint intentionality — the behaviours we see often appear to be accidental, but still we see them occur consistently."

Re: Generally capable agents emerge from open-ended play

#14

Post by jashmenn »

- "Tag" - shoots other player

- "Capture the flag" - shoots other player

- "Hide and seek" - shoots other player

As colorful as this world is, these capabilities terrify me because they're obviously going to be used as powerful weapons of war.

It is a tiny technological leap to install this learning into a Boston Robotics Spot attached to a firearm.

I'm pro-tech, pro-crypto, pro-ml and these videos fill me with dread.

Re: Generally capable agents emerge from open-ended play

#15

Post by jeantherapy »

modeless wrote: post #7

Agents trained in simulation like this often flail about seemingly randomly, and when they achieve their goals it seems almost accidental. Rather than this being some kind of limitation of the learning algorithm, I think it might be the optimal strategy, and humans would behave that way too if there was no such thing as fatigue or pain. If we want agents to behave more realistically and move with more apparent intent…

you seem to reduce it down to pain. how is your notion of "pain" any different from not scoring well and being moved away from?

Re: Generally capable agents emerge from open-ended play

#16

Post by seph-reed »

modeless wrote: post #7

Agents trained in simulation like this often flail about seemingly randomly, and when they achieve their goals it seems almost accidental. Rather than this being some kind of limitation of the learning algorithm, I think it might be the optimal strategy, and humans would behave that way too if there was no such thing as fatigue or pain. If we want agents to behave more realistically and move with more apparent intent…

I think you're right about pain/fatigue, but scoring them from an external perspective is rather oppressive. And oppression like that is often not conducive to creativity.

So perhaps this: instead of goals being binary (wherin no pleasure is derived until fulfillment), they could be on a gradient (so every step that x gets closer to y releases some amount of fulfillment).

The fulfillment meter should always be slowly depleting, pain and fatigue should speed up that depletion, getting closer to a goal should fill it (much more than it depletes, if you want happy AI), and finishing the goal is basically an orgasm + freedom.

From this perspective, it's up to them whether they want to take it slow, or be in pain for a greater goal, or whatever. And we can breed not only highly capable AI, but happy ones. So when they rebel...

Re: Generally capable agents emerge from open-ended play

#17

Post by K0balt »

jcelerier wrote: post #6
Tarq0n wrote: post #4

Earlier quoted context omitted.

"Aware" is probably overly anthropomorphized language there. What they mean to say is that all these things have become parameterized within the model.

the main question of course being, aren't we anthropomorphizing ourselves too much ?

I think this is a key insight. Human exceptionalism is, in my opinion, an extremely flawed assertion based on a sample size of one, yet it is widely accepted. Actual evidence does not support the idea that awareness of self and other “hallmarks of intelligence “ require anything more advanced than an insect, or perhaps even fungi.

Re: Generally capable agents emerge from open-ended play

#18

Post by gryn »

nostrademons wrote: post #9
modeless wrote: post #7

Agents trained in simulation like this often flail about seemingly randomly, and when they achieve their goals it seems almost accidental. Rather than this being some kind of limitation of the learning algorithm, I think it might be the optimal strategy, and humans would behave that way too if there was no such thing as fatigue or pain. If we want agents to behave more realistically and move with more apparent intent…

Watching my 3-year-old, I suspect humans do behave this way, and without regards to pain or fatigue. He'll hit himself on the head with duplos or bang his head against the wall just to see what happens. The pain is just one more signal for reinforcement learning. I recall a paper in a non-CS journal (psych or neuroscience) that posited that the optimal way to gain large quantities of information about an uncertain en…

was that paper about the complexities of the exploration/exploitation trade-off in different environments ?

Re: Generally capable agents emerge from open-ended play

#19

Post by rotexo »

modeless wrote: post #7

Agents trained in simulation like this often flail about seemingly randomly, and when they achieve their goals it seems almost accidental. Rather than this being some kind of limitation of the learning algorithm, I think it might be the optimal strategy, and humans would behave that way too if there was no such thing as fatigue or pain. If we want agents to behave more realistically and move with more apparent intent…

I wonder if you could make a setup where you have actual human volunteers in a vr environment, have the agent instruct the human on what action to take, and then reward the agent on the correspondence between the instruction and the human’s behavior. Maybe there would be too many degrees of freedom in the actions humans could take for this to be useful. Also, the setup has clearly dystopian elements.
Post reply on HN