Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Agent Cooperation

deepmind.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Understanding Agent Cooperation

#11

Post by wukerplank »

tutts wrote: post #6
leereeves wrote: post #4

Earlier quoted context omitted.

I think aggressive is a better (more descriptive, narrower) word than compete here. Two racers are competing to see who runs faster, but if one pulls out a laser gun and shoots the other, that's aggressive.

Not if laser guns are a part of the race. Then it's just competitive.

If the laser gun is not part of the race, it's cheating (at best). I don't understand this nitpicking. Of course a gun is an aggressive tool. "Aggressive" and "competitive" are not mutually exclusive.

Re: Understanding Agent Cooperation

#12

Post by jfoster »

leereeves wrote: post #4
bencollier49 wrote: post #3

What an awful headline. "AI learns to compete in competitive situations" should be the precis. Basically, it learned that it didn't need to fight until there was resource scarcity in a simulation.

I think aggressive is a better (more descriptive, narrower) word than compete here. Two racers are competing to see who runs faster, but if one pulls out a laser gun and shoots the other, that's aggressive.

To the AI, there is nothing aggressive about the "laser gun"; as far as it was aware, the "laser gun" could be any tool. It was just doing what it had determined may help it achieve a better score.

Re: Understanding Agent Cooperation

#14

Post by JamilD »

The AI can minimize loss / maximize fitness by either moving to look for additional resources, or fire a laser.

Turns out that when resources are scarce, the optimal move is to knock the opponent away. I think this tells us more about the problem space than the AI itself; it's just optimizing for the specific problem.

Re: Understanding Agent Cooperation

#15

Post by leereeves »

jfoster wrote: post #12
leereeves wrote: post #4

Earlier quoted context omitted.

I think aggressive is a better (more descriptive, narrower) word than compete here. Two racers are competing to see who runs faster, but if one pulls out a laser gun and shoots the other, that's aggressive.

To the AI, there is nothing aggressive about the "laser gun"; as far as it was aware, the "laser gun" could be any tool. It was just doing what it had determined may help it achieve a better score.

The "laser gun" knocks the other player out of the race.

It's not the name that makes me consider it aggressive, rather the fact that it works by harming the other player.

It's probably true the AI doesn't distinguish "aggressive" tools from other tools. Isn't that one of lessons here? If an AI isn't taught not to be aggressive, it will choose to harm other participants when that's the most effective strategy.

Re: Understanding Agent Cooperation

#16

Post by katzgrau »

The article at first suggests that more intelligent versions of AI led to greed and sabotage.

But I do wonder if an even more intelligent AI (perhaps in a more complex environment) would take the long view instead and find a reason to co-habitate.

It's kind of like rocks, paper scissors - when you attempt to think several levels deeper than your opponent and guess which level they stopped at. At some intelligence level for AI, cohabitation seems optimal - at the next level, not so much, and so on.

We're probably going to end up building something so complex that we don't quite understand it and end up hurting somebody.

Re: Understanding Agent Cooperation

#18

Post by vanderZwan »

You know, rather than being scared by this, I think it's an excellent opportunity to learn how and when aggression evolves, and maybe learn how we can set up systems that nudge people to collaborate, perhaps even when resources are scarce.

Re: Understanding Agent Cooperation

#20

Post by MatekCopatek »

leereeves wrote: post #10
itsdrewmiller wrote: post #7

Earlier quoted context omitted.

The game included the ability to shoot the other player with no consequence - it sounds aggressive because "laser beam" but if you called it "tagging" then it would more clearly just be an in-rules option.

I don't think "in-rules" and "aggressive" are mutually exclusive. It's fair to call blitzing the QB an aggressive move in American football.

You're technically correct, but the football analogy switches context so that the meaning of aggressive is no longer bad.

I think the point bencollier49 is trying to make is that we simply gave software a specific set of rules to train it. It doesn't know how we perceive the actions it is performing.

The game could be described as two people eating poisonous apples in order to prevent the other person from dying. In that case, the currently greedy one would be the hero.

Post reply on HN