Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Agent Cooperation

deepmind.com

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Understanding Agent Cooperation

#41

Post by projektir »

Kalium wrote: post #40
projektir wrote: post #26

Earlier quoted context omitted.

I'm assuming you're referring to something like this: https://egtheory.wordpress.com/2015/03/02/ipd/ I think we shouldn't confuse efficient strategies with the chosen strategies. What causes Moloch is the inability to see the big picture, to see outside of the self in the collective (maybe Buddhism has a point). An efficient strategy may very well be something we'd prefer, such as tit-for-tat. But is that the strateg…

In the long run, we've built massively complex human societies that develop intricate technologies. Technologies whose production requires supply chains many thousands of people, and so complex that nobody involved understands all the technologies involves. All so some people can contend that humanity isn't able to see the collective beyond the self. I would say we have a demonstrated ability of seeing the big pictur…

> In the long run

Hence I said:

> without a lot of time

An AI spending a lot of time doing effectively the same thing humans have been doing (read: propagation of immense amounts of suffering) is not really something I'd want to see repeated. It seems rather obvious that these conclusions are very difficult and slow to arrive at at a proper scale, so no AI will have them by default. They'll be aggressive by default, just like your average animal in evolution. The fact that given their own millions of years (sped up) they may eventually arrive at the rudimentary level of cooperation that humans possess does not instill a lot of hope in me.

> I would say we have a demonstrated ability of seeing the big picture, and a pretty good track record of making it work.

I'm talking about this: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

A good example that's going to be hard to ignore will be the upcoming climate change due to humans catastrophically failing to see the big picture and focusing on smaller gains within their sub-groups. It really doesn't have much to do with complexity, but it has everything to do with the very same behavior you're seeing the AI execute here.

Re: Understanding Agent Cooperation

#42

Post by lawless123 »

projektir wrote: post #41
Kalium wrote: post #40

Earlier quoted context omitted.

In the long run, we've built massively complex human societies that develop intricate technologies. Technologies whose production requires supply chains many thousands of people, and so complex that nobody involved understands all the technologies involves. All so some people can contend that humanity isn't able to see the collective beyond the self. I would say we have a demonstrated ability of seeing the big pictur…

> In the long run Hence I said: > without a lot of time An AI spending a lot of time doing effectively the same thing humans have been doing (read: propagation of immense amounts of suffering) is not really something I'd want to see repeated. It seems rather obvious that these conclusions are very difficult and slow to arrive at at a proper scale, so no AI will have them by default. They'll be aggressive by default,…

I cant help but notice you and someone else are here both promoting this "Moloch" stuff and that particular website slatestarcodex

https://news.ycombinator.com/reply?id=13636150&goto=threads%...

why?

Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology.

Re: Understanding Agent Cooperation

#43

Post by Sacho »

projektir wrote: post #33
philipov wrote: post #27

Earlier quoted context omitted.

> Looking at the long history of evolution, I'd say no. This entire lecture series on Human Behavioral Biology is worth watching from the beginning, but I've linked to a moment where Sapolsky describes tit-for-tat strategies arising in animals. First example: Vampire Bats. []: https://www.youtube.com/watch?v=Y0Oa4Lp5fLE&feature=youtu.be...

I think we must be looking at different scales, and I'm probably mixing up terms and not making myself very clear. Evolution defaults to aggression as that is how it squeezes out fitness, and cooperative behavior is continually at odds with that and only seems to survive on one level up every so often, where evolution just starts treating it as giant agents anyway and the cycle starts again at a higher level. Similar…

I don't know what you mean when you say "evolution". Certainly, the evolution of mammals shows cooperation as a common, viable strategy - just look at all the species operating in packs/flocks/herds. The little I know of simple organism evolution also seems to suggest cooperation(symbiosis) is an important part in evolving into a complex organism.

But the best part about evolution is that we don't need to replicate blind mutation and strict fitness functions, we can use the proven-to-work strategies as our springboard. And the best part about AI is that we have no ethical issues simulating millions of evolutionary iterations of "bloodshed" until we arrive at an AI that is acceptable to our ethics.

Re: Understanding Agent Cooperation

#44

Post by projektir »

lawless123 wrote: post #42
projektir wrote: post #41

Earlier quoted context omitted.

> In the long run Hence I said: > without a lot of time An AI spending a lot of time doing effectively the same thing humans have been doing (read: propagation of immense amounts of suffering) is not really something I'd want to see repeated. It seems rather obvious that these conclusions are very difficult and slow to arrive at at a proper scale, so no AI will have them by default. They'll be aggressive by default,…

I cant help but notice you and someone else are here both promoting this "Moloch" stuff and that particular website slatestarcodex https://news.ycombinator.com/reply?id=13636150&goto=threads%... why? Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology.

SSC, while fairly controversial (and I strongly disagree with a LOT of what's on there), is mostly known to HN. At the moment, it has the best summary of the concept that I'm aware of.

> Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology.

Well, frankly, I disagree. Humans have an inherent blind spot when it comes to complex systemic forces. We tend to imagine them as weak and irrelevant. Reframing them as villains seems to be necessary to understand their power and reach.

Re: Understanding Agent Cooperation

#45

Post by dang »

tdkl wrote: post #17
bencollier49 wrote: post #3

What an awful headline. "AI learns to compete in competitive situations" should be the precis. Basically, it learned that it didn't need to fight until there was resource scarcity in a simulation.

Progressive left double speak, nothing new.

Please don't post unsubstantive comments, and especially don't do partisan name-calling here. It destroys the kind of discussion HN exists for.

Edit: unfortunately you've been doing this a lot. We ban accounts that do this, so please stop.

Re: Understanding Agent Cooperation

#46

Post by dang »

saycheese wrote: post #21

This reads as click-bait, here's the original blog post and research paper by DeepMind: "Understanding Agent Cooperation" https://news.ycombinator.com/edit?id=13635218

Thanks! We changed to that from http://www.sciencealert.com/google-s-new-ai-has-learned-to-b....

Re: Understanding Agent Cooperation

#47

Post by eduren »

Sacho wrote: post #43
projektir wrote: post #33

Earlier quoted context omitted.

I think we must be looking at different scales, and I'm probably mixing up terms and not making myself very clear. Evolution defaults to aggression as that is how it squeezes out fitness, and cooperative behavior is continually at odds with that and only seems to survive on one level up every so often, where evolution just starts treating it as giant agents anyway and the cycle starts again at a higher level. Similar…

I don't know what you mean when you say "evolution". Certainly, the evolution of mammals shows cooperation as a common, viable strategy - just look at all the species operating in packs/flocks/herds. The little I know of simple organism evolution also seems to suggest cooperation(symbiosis) is an important part in evolving into a complex organism. But the best part about evolution is that we don't need to replicate b…

> And the best part about AI is that we have no ethical issues simulating millions of evolutionary iterations of "bloodshed" until we arrive at an AI that is acceptable to our ethics.

Are we sure this is the case? Once we start attempting to create an AI that follows our modern ethics, we have to start asking questions about AI personhood. And I for one feel there are deep ethical questions regarding forced iteration/simulation, let alone the violent kind.

Re: Understanding Agent Cooperation

#48

Post by SpaceManiac »

lawless123 wrote: post #42
projektir wrote: post #41

Earlier quoted context omitted.

> In the long run Hence I said: > without a lot of time An AI spending a lot of time doing effectively the same thing humans have been doing (read: propagation of immense amounts of suffering) is not really something I'd want to see repeated. It seems rather obvious that these conclusions are very difficult and slow to arrive at at a proper scale, so no AI will have them by default. They'll be aggressive by default,…

I cant help but notice you and someone else are here both promoting this "Moloch" stuff and that particular website slatestarcodex https://news.ycombinator.com/reply?id=13636150&goto=threads%... why? Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology.

It's not specifically new technology that's being deliniated by "Moloch". The article is long but explains clearly the (not entirely new) association: Moloch represents the sacrifice of human values on the altar of competitiveness, and is best staved off through coordination.

Re: Understanding Agent Cooperation

#49

Post by [deleted] »

projektir wrote: post #44
lawless123 wrote: post #42

Earlier quoted context omitted.

I cant help but notice you and someone else are here both promoting this "Moloch" stuff and that particular website slatestarcodex https://news.ycombinator.com/reply?id=13636150&goto=threads%... why? Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology.

SSC, while fairly controversial (and I strongly disagree with a LOT of what's on there), is mostly known to HN. At the moment, it has the best summary of the concept that I'm aware of. > Frankly i don't feel it's productive or rational to attach the name of a biblical villain to new technology. Well, frankly, I disagree. Humans have an inherent blind spot when it comes to complex systemic forces. We tend to imagine t…

[deleted]

Re: Understanding Agent Cooperation

#50

Post by laretluval »

JamilD wrote: post #14

The AI can minimize loss / maximize fitness by either moving to look for additional resources, or fire a laser. Turns out that when resources are scarce, the optimal move is to knock the opponent away. I think this tells us more about the problem space than the AI itself; it's just optimizing for the specific problem.

> I think this tells us more about the problem space than the AI itself; it's just optimizing for the specific problem.

My reading is that the point of this research is to find out what problem spaces are conducive to cooperation, not to find out details of how these particular agents work.

Post reply on HN