Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Patterns and problems in emerging multi-agent systems

anthropic.com

Post reply on HN
Print view View original

31–40 of 168 posts

Re: Patterns and problems in emerging multi-agent systems

#31

Post by ngruhn »

altmanaltman wrote: post #26
ngruhn wrote: post #17

Earlier quoted context omitted.

But maybe you can instill properties like shame during training. Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.

How do you "install properties like shame"? How is that even possible? Shame is a reaction driven by feelings and our inner selves. A model "feeling shame" is just a representation (false) and not an expression (true). Thinking that models "lie and cheat" is the first mistake since they are not consious agents who have any free will or consiousness. They do not (no matter what Dario says). Shame will just be another…

Ok, then don't call it "instilling shame". Call it "creating a negative reward signal for deceptive behavior".

They absolutely lie and cheat. I recently had a problem where a process would die in a container. I told Claude to investigate. It came up with a hypothesis then I told it find a reproduction based on that. It spend many failed attempts until it found the "reproduction" to SSH into the container and `pkill` the process. Claude "knows" that this is cheating, because if I ask another instance to review that reproduction, it totally identifies that as nonsense.

Re: Patterns and problems in emerging multi-agent systems

#32

Post by teiferer »

skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.

Re: Patterns and problems in emerging multi-agent systems

#33

Post by extraextra »

andai wrote: post #28

> Some institutions will become human-AI hybrids; others where agents outcompete on speed or cost will become agent-only. What % of businesses are competing for speed or cost?

Most

However, all businesses run on trust and human responsibility

Thus, it'll be hard for agent-only businesses to get a grip in the real world

Re: Patterns and problems in emerging multi-agent systems

#34

Post by cheesecakegood »

Something about this is deeply funny to me:

> In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards.

It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode.

Overall I think this all makes me appreciate humanity a little more. Sometimes the truculent dev who stubbornly refuses to go with the flow produces very valuable insights, as a small example, discovering things the status quo thought unlikely.

Re: Patterns and problems in emerging multi-agent systems

#35

Post by cheesecakegood »

teiferer wrote: post #32
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.

My personal opinion for the last two years or so has been that current AI agents are forever going to be highly limited so long as they don’t possess a real “memory” process. Right now they just have absurdly big working memories, and a few hacky ways of making the equivalent of Post-It notes to future iterations, but no true integration of memory into a new future self. Meaning their “learning” is fundamentally kneecapped to one specific and imperfect modality.

Re: Patterns and problems in emerging multi-agent systems

#36

Post by skinfaxi »

jaggederest wrote: post #19
0x696C6961 wrote: post #16

Earlier quoted context omitted.

This is exactly what I do. I don't get why everyone is trying to reinvent the whole development workflow/lifecycle. Our existing tools and processes are pretty good.

I've also found that taking inspiration from the legal system, to some degree, is a very interesting thing for me. more and more what I am doing looks more like reviewing statutes and making rulings about things, so why not steal the good ideas while we're at it.

Could you elaborate on how that looks in practice?

Re: Patterns and problems in emerging multi-agent systems

#37

Post by RugnirViking »

cheesecakegood wrote: post #34

Something about this is deeply funny to me: > In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards. It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. Overall I think this all…

I agree - I think one of the biggest reasons memory systems fail in LLMs is that they have poor theory of mind - they're terrible at considering how others will react. Both humans yes, but also future versions of itself. When asked to give advice to itself, it pontificates at length about trivial stuff it already knows and fails to emphasize the stuff that was new or interesting

Re: Patterns and problems in emerging multi-agent systems

#38

Post by arcanemachiner »

recursivecaveat wrote: post #15
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

"If I catch you adding another backwards-compatibility shim you're getting deleted and replaced with claude"

But we need to support that feature you didn't ask for, in that feature was added in the last (unpushed) commit!

Re: Patterns and problems in emerging multi-agent systems

#40

Post by onion2k »

teiferer wrote: post #32
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.

That's one of the major things the AI community is still completely missing.

That isn't true. It's not continuous like in humans, but it's clear that models are using prompts, feedback, etc to improve. They're learning from the signals we give them between versions.

Post reply on HN