Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emergent Tool Use from Multi-Agent Interaction

openai.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: Emergent Tool Use from Multi-Agent Interaction

#41

Post by benrbray »

breck wrote: post #40
vertoc wrote: post #33

Earlier quoted context omitted.

Right but to fully understand what's going on you need to also visualize the physics in a 3D world - just dots and lines and squares wouldn't fully show what's going on. This may be close to the simplest visualization that made sense

The visualizations look great, but wouldn't run on an N64, which had many physics games. I'm wondering the same thing as the OP--was this advanced level of graphics used during the research, or was the styling added after the fact for readers? A low res visualization seems like it would do the job equally well, but maybe not. Curious what they are finding and whether there are benefits to having a great looking visua…

Researchers have much better graphics tools available to them today than they did in the N64 era. Basic familiarity with e.g. Unity would be enough to run these sorts of simulations.

Re: Emergent Tool Use from Multi-Agent Interaction

#42

Post by The_Amp_Walrus »

The_rationalist wrote: post #39

Am I misunderstanding something? Instead of teaching the "AI" intelligent rules or rules for creating rules for maximising their goals. They teach them nothing, which means they have 0 usable high level knowledge. And the "AI" pure bruteforce for finding empirically best solutions for this ridiculously simple universe. How is that advancing research? This is just a showcase of what modern hardware can do, and also a…

Some AI researchers believe that using learning methods with no built-in prior knowledge and throwing a bunch of compute at them is the path to building effective AI. I'm thinking of Richard Sutton in particular:

- Bitter Lesson essay: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

- A lecture of his on temporal difference learning, which is a "model-free" method of reinforcement learning: https://www.youtube.com/watch?v=LyCpuLikLyQ

I personally don't agree with his emphasis on model-free learning, but it's not the case that people are building model-free RL agents because they don't understand the trade off that they're making.

Re: Emergent Tool Use from Multi-Agent Interaction

#43

Post by PhasmaFelis »

vertoc wrote: post #33
boardwaalk wrote: post #32

Earlier quoted context omitted.

They're talking about the visualization, not the physics. The agents aren't getting visual input. That would make things much, much slower.

Right but to fully understand what's going on you need to also visualize the physics in a 3D world - just dots and lines and squares wouldn't fully show what's going on. This may be close to the simplest visualization that made sense

An accurate 3D visualization could have been a lot simpler than this. The actors are most likely modeled physically as simple cylinders; all the character animations are extraneous. And there's plenty of subtle effects in the seekers' vision cones, the reflective floor, the uneven box landscape outside, etc.

So, as tlb said, I'm curious if all of that was added for the public release, or if the researchers set it up while running the experiments. It seems like it would be fun.

Re: Emergent Tool Use from Multi-Agent Interaction

#44

Post by PhasmaFelis »

CodeGlitch wrote: post #19
bryanrasmussen wrote: post #12

Earlier quoted context omitted.

hmm, yes in the story I'm envisioning the AIs don't wipe out humanity because they have achieved sentience, but just because it turns out killing all humans is an optimizing component of solving some other problem.

I think we humans have already solved this problem you describe... we call them laws. We use these laws to prevent people doing bad things , and I see no reason why they can't be described to an AI to drive its behavior to one that isn't going to end humanity . for the most part. fingers crossed.

I think you're misunderstanding the problem. Expressing complex rules in a machine-readable format is the least of the issues here. The main problem is that training AIs to optimize certain behaviors within constraints very frequently leads to them accidentally discovering "loopholes" that would never have occurred to a human (as with "box surfing" here). The AI doesn't know it's "cheating"; its behavior may be emergently complex, but its model of our desires is only what we tell it.

A naive and unlikely example would be telling an AI to maximize human happiness and prevent human harm, so it immobilizes everyone and sticks wires into their pleasure centers. Everyone is as happy as it is possible for a human to be, and no one is doing anything remotely dangerous!

The actual dangers will be stranger and harder to predict. I'm not saying we can't find a way to make strong AI safe. I'm just saying that it's a much trickier task than you imply.

https://www.wired.com/story/when-bots-teach-themselves-to-ch...

https://vkrakovna.wordpress.com/2018/04/02/specification-gam...

Re: Emergent Tool Use from Multi-Agent Interaction

#45

Post by jcims »

SmooL wrote: post #3

Amazing. Very cool to see this sort of emergent behavior. I also very much enjoyed this section: "We propose using a suite of domain-specific intelligence tests that target capabilities we believe agents may eventually acquire. Transfer performance in these settings can act as a quantitative measure of representation quality or skill, and we compare against pretraining with count-based exploration as well as a traine…

Did you see the 'Surprising Behaviors' at the bottom? Pretty funny

https://openai.com/blog/emergent-tool-use/#surprisingbehavio...

Re: Emergent Tool Use from Multi-Agent Interaction

#46

Post by smogcutter »

Falling3 wrote: post #35
ismail wrote: post #15

Earlier quoted context omitted.

Asimov 3 rules as the final policy when making decisions should sort this problem out. This assumes that the rules cannot be changed by the AI.

I've always been very incredulous that there would be any possibility of taking something sufficiently complex to be considered an AGI and hard-coding anything like the 3 rules into it.

By the same token, I’m extremely suspicious of the idea that such a sufficiently complex AGI could also be dumb enough to optimize for paper clip production at the expense of all life on earth (or w/e example).

Re: Emergent Tool Use from Multi-Agent Interaction

#47

Post by lopmotr »

brianpgordon wrote: post #11

This is incredible. The various emergent behaviors are fascinating. I remember being amazed a decade ago by the primitive graphics in artificial life simulators like Polyworld: https://en.wikipedia.org/wiki/Polyworld https://www.youtube.com/watch?v=_m97_kL4ox0&t=9m43s It seems that OpenAI has a great little game simulated for their agents to play in. The next step to make this even cooler would be to use physical, ro…

> real meatspace!

I'm doing something like this as a hobby but only single agent. The input is camera images and reward is based on a stopped/moving flag determined by changes between successive images as well as favoring going forward over turning. So far, it can learn to avoid crashing into walls, which is about all I'd expect. Trying to find good automated rewards without building too much special hardware is difficult. It's a vanilla DQN.

Re: Emergent Tool Use from Multi-Agent Interaction

#48

Post by teabee89 »

haylel wrote: post #27

Looks awesome. I tried coding up a multi-agent system for my CS degree and it was incredibly complicated. I was trying to implement an algorithm I found to give each agent emotions of fear, anger, happiness and sadness in order to change their behaviours... it was way more difficult than I expected but you can read more about it here if you're also interested in this stuff. The 3D graphics in this example are way coo…

This is really interesting, thank you for sharing!

Re: Emergent Tool Use from Multi-Agent Interaction

#49

Post by lettergram »

Even my work with basic circuits for sea slugs led to “cooperative” behavior:

https://austingwalters.com/modeling-and-building-robotic-sea...

I think sometimes we see what we want to see. Not saying it’s not interesting work, just that it’s less round breaking than you may think.

Re: Emergent Tool Use from Multi-Agent Interaction

#50

Post by godelski »

corey_moncure wrote: post #7

One plausible, perhaps optimal strategy in the second arena is for the hiders to build a shelter around the seekers and lock them in place, circumventing the whole cat and mouse over ramps and ramp surfing (which the seekers would never be able to access). I wonder why this strategy is not arrived at.

> I wonder why this strategy is not arrived at.

That's always a good question! One thing to remember is that in RL you are looking through large solution spaces. You probably aren't going to find a global optima (if one even exists!). What will happen is that a local optima is found and just one that _works_. This is why having a feature rich space is important, because it helps you escape the locality, but also remember that we don't even know what the solution space looks like and what an optimal solution is.

It is also entirely possible that you retrain something from scratch and find a different local optima. Self play can help with this as well as multi-agents, but we're still not guaranteed to find every solution nor solutions that appear obvious to us. RL just tries things (often randomly) till they start working (then they bias towards what worked).

Post reply on HN