Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emergent Tool Use from Multi-Agent Interaction

openai.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Emergent Tool Use from Multi-Agent Interaction

#51

Post by ludwigschubert »

smogcutter wrote: post #46
Falling3 wrote: post #35

Earlier quoted context omitted.

I've always been very incredulous that there would be any possibility of taking something sufficiently complex to be considered an AGI and hard-coding anything like the 3 rules into it.

By the same token, I’m extremely suspicious of the idea that such a sufficiently complex AGI could also be dumb enough to optimize for paper clip production at the expense of all life on earth (or w/e example).

...and many would say that’s because us humans are bad at imagining optimizing agents without anthropomorphizing them. This is a reasonable, even typical suspicion that many people share! The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video, but if you prefer a more thorough treatment, you could also read about “instrumental convergence” directly. If you find a flaw in this idea, I’d be interested to hear about it! :)

Robert Miles’ video: https://youtu.be/ZeecOKBus3Q

Instrumental Convergence: https://arbital.com/p/instrumental_convergence/

Now afaik nothing in this argument says that we can’t find a way to control this in a more complex formalism-but we clearly haven’t done so yet.

Re: Emergent Tool Use from Multi-Agent Interaction

#52

Post by dooglius »

The state space here looks pretty small, it seems to me that with so much training it's just a case of brute-force search. When I think of "tool use" in regards to the intelligence of early humans, I imagine something more like [0] where the state space is enormous and it takes a good deal of reasoning and planning to get to a desired result.

[0] https://www.youtube.com/watch?v=BN-34JfUrHY

Re: Emergent Tool Use from Multi-Agent Interaction

#53

Post by cscurmudgeon »

gdb wrote: post #38
Inufu wrote: post #36

Nice visualizations and explanation! You might want to make it clearer that the agents don't actually receive any visual observations, but rather directly the xy positions of all other agents and objects. This also seems very similar to "Capture the Flag: the emergence of complex cooperative agents" ( https://deepmind.com/blog/article/capture-the-flag-science )? Regarding the conclusion: > We’ve provided evidence tha…

Good catch! Will update the post to be explicit that there are many pre-existing awesome results in this vein.

Any possibility of releasing the simulation environment? Looks quite cool!

Re: Emergent Tool Use from Multi-Agent Interaction

#54

Post by shubidubi »

gdb wrote: post #16
jpetrucc wrote: post #14

As always, crazy interesting stuff coming out of OpenAI! This is the type of stuff that amazes me - I really wish I had more of an opportunity to play with AI/ML in my day to day work.

> I really wish I had more of an opportunity to play with AI/ML in my day to day work. Anyone who feels this way — we're hiring :)! https://openai.com/jobs/ . (Also if I can answer any questions about OpenAI, feel free to ping me at gdb@openai.com.)

Anything remote friendly? Unfortunately all jobs are in SF only.

Re: Emergent Tool Use from Multi-Agent Interaction

#56

Post by nitrogen »

The_rationalist wrote: post #39

Am I misunderstanding something? Instead of teaching the "AI" intelligent rules or rules for creating rules for maximising their goals. They teach them nothing, which means they have 0 usable high level knowledge. And the "AI" pure bruteforce for finding empirically best solutions for this ridiculously simple universe. How is that advancing research? This is just a showcase of what modern hardware can do, and also a…

How do you know your own brain isn't running thousands of parallel simulations in your head, even though you perceive it only once? How did your brain learn to reason about physics in the first place if not by repeatedly finding objects in your environment and randomly manipulating them?

Re: Emergent Tool Use from Multi-Agent Interaction

#57

Post by nicklovescode »

dooglius wrote: post #52

The state space here looks pretty small, it seems to me that with so much training it's just a case of brute-force search. When I think of "tool use" in regards to the intelligence of early humans, I imagine something more like [0] where the state space is enormous and it takes a good deal of reasoning and planning to get to a desired result. [0] https://www.youtube.com/watch?v=BN-34JfUrHY

It's unclear to me that we navigated the state space so discretely. My guess would be that we used a combination of rock throwing + stock hitting before eventually deciding that combining the two might be fruitful.

After the idea is polished it looks clever, but it may have been invented through a series of mostly random steps

Re: Emergent Tool Use from Multi-Agent Interaction

#60

Post by YeGoblynQueenne »

This is visually very impressive, of course, but what is the significance of this work? I am not very familiar with intelligent agents research so I don't understand to what extent learning cooperative tool use in an adversarial environment (if I understand correctly what is shown) represents an important advancement of the state of the art in intelligent agents research, or not.

In any case this is a simulation- so it's basically impossible to take the learned model and use it immediately in a real-world environment with true physics and arbitrary elements, let alone with unrestricted dimensions (the agents in the article are for the most part restricted to a limited play area). So if I understand this correctly the trained model is only good for the specific simulated environment and would not work as well under even slightly different conditions.

Post reply on HN