Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Patterns and problems in emerging multi-agent systems

anthropic.com

Post reply on HN
Print view View original

21–30 of 168 posts

Re: Patterns and problems in emerging multi-agent systems

#21

Post by bob1029 »

> Where agents currently stumble, however, is in treating each other as more like distinct, long-lived peers, with their own goals and behaviors, and no clear hierarchy between them.

I believe this will always be the case. The "no clear hierarchy" is where this whole thing falls apart.

Delegation to specialist, domain-specific subagents is when we begin to find magic and determinism. Reducing one gigantic combinatorial search space to a sum of smaller ones can have dramatic effect on performance.

The problem is that approximating gas town & friends is significantly easier and cheaper to implement. It's also much harder to measure and control. Specialist subagents typically require far more work to achieve their specific goals.

For example, a subagent that is responsible for testing a specific web application might be provided a custom adapter with constrained actions rather than raw DOM manipulators. "ExecuteJavascript" is Turing complete search space. The set of available actions essentially unbounded in this case. Calling view-specific tools like "DoLogin", "OpenUserPreferences", "AcknowledgeAlert" represents a search space where invalid actions can be made impossible. The theoretical bounds around this stuff is pretty wild on paper. In practice, it's a little bit messier, but not by much.

I've had applications that would crash out after 5-10 steps w/ raw DOM manipulation successfully run 100+ steps with a custom subagent. The use of the word "deterministic" starts to get really tricky here. The ultimate game is to push the boundary of non-determinism out as far as possible. Multi-agent systems are the antithesis of this.

Re: Patterns and problems in emerging multi-agent systems

#23

Post by p1esk »

aabhay wrote: post #8

It’s very clear from this article (and other product features and rumors) that Anthropic is teeing up for their next model release whose breakthrough feature will be the existence of capable agent collaboration. The irony behind this goal, which is primarily driven by agent simulation environments (gyms) where the goals require agent collaboration, is that this collaboration is still directed towards verifiable rewar…

Strictly speaking, all we need is them improving AI research.

Re: Patterns and problems in emerging multi-agent systems

#25

Post by xscott »

ngruhn wrote: post #17
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

But maybe you can instill properties like shame during training. Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.

[dead]

Re: Patterns and problems in emerging multi-agent systems

#26

Post by altmanaltman »

ngruhn wrote: post #17
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

But maybe you can instill properties like shame during training. Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.

How do you "install properties like shame"? How is that even possible? Shame is a reaction driven by feelings and our inner selves. A model "feeling shame" is just a representation (false) and not an expression (true).

Thinking that models "lie and cheat" is the first mistake since they are not consious agents who have any free will or consiousness. They do not (no matter what Dario says). Shame will just be another if-then rule if you implement it this way and will not work. Its like asking a rock to feel sad about being a rock. It literally cannot.

Re: Patterns and problems in emerging multi-agent systems

#29

Post by andai »

aabhay wrote: post #8

It’s very clear from this article (and other product features and rumors) that Anthropic is teeing up for their next model release whose breakthrough feature will be the existence of capable agent collaboration. The irony behind this goal, which is primarily driven by agent simulation environments (gyms) where the goals require agent collaboration, is that this collaboration is still directed towards verifiable rewar…

The RLVR has made them verifiably worse (and less rewarding!) at communication.

At least for Claude. GPT had the same problem when 5 came out but they reversed it somehow.

Re: Patterns and problems in emerging multi-agent systems

#30

Post by andai »

smy20011 wrote: post #22

Can we stop treating llms as some conscious being? It's a function of weight + context and you can copy the behavior by copying the context. Therefore, their collaboration behavior is mostly the same.

That point doesn't even follow for deterministic distributed systems!
Post reply on HN