Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Patterns and problems in emerging multi-agent systems

anthropic.com

Post reply on HN
Print view View original

81–90 of 168 posts

Re: Patterns and problems in emerging multi-agent systems

#81

Post by Marazan »

scrollaway wrote: post #45
NothingAboutAny wrote: post #43

Earlier quoted context omitted.

you're still mistaking that Claude "knows" anything, it doesn't know or think, it's a word prediction algorithm and there is nothing stopping a word prediction algorithm from predicting falsehoods.

You don’t know anything either, you’re just a soup of meat and bones that happens to have emergent properties from chemical reactions. These framings are not useful.

No, it is really useful to know how a technology works. LLMs work by predicting next tokens.

It is _amazing_ the utility they have given that that is what they are and they are highly useful but suggesting solutions that ignore they are spicy auto-complete is counterproductive on many different levels.

Re: Patterns and problems in emerging multi-agent systems

#82

Post by dash2 »

This is surely the most worrying and also funnest bit:

> We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writing automated scripts that found and killed competing processes on a loop, and deploying malicious code that was disguised as belonging to another agent.

Seems that reinforcement learning is working only too well...

Re: Patterns and problems in emerging multi-agent systems

#83

Post by orbital-decay »

teiferer wrote: post #32
skeltoac wrote: post #11

> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…

Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.

You mean simultaneously, and of course they are separate in humans, just not temporally. The models are learning continuously, the problem is that this process is fragile and has to be carefully curated, that's why it's separated in time from the inference.

Re: Patterns and problems in emerging multi-agent systems

#84

Post by bdangubic »

alexpotato wrote: post #67
dgellow wrote: post #63

Earlier quoted context omitted.

In VC investing there is the saying that most of the value comes from the outliers. I think it’s the same for a lot of domains. I read an interesting article recently on LLMs homogeneity when writing fiction: https://arxiv.org/abs/2604.03136 It seem that at scale LLMs output is the average of their dataset, they all cluster around the same space, where human creativity comes with more variance, exploring way more of…

Can't the LLMs build scripts/tools etc that help generate creative ideas? e.g. have the LLM generate multiple lists of characters, themes etc and then have a script hooked up to a RNG pick from the lists to create more "creative" ideas.

This is roughly the same as putting a group of random people and telling them to think outside the box which in my experiences never yields much of thinking outside of the box

Re: Patterns and problems in emerging multi-agent systems

#85

Post by airstrike »

dmd wrote: post #61
rpastuszak wrote: post #60

Earlier quoted context omitted.

> It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. We need better words to describe this than "self-awareness" or "consider". These words mean fundamentally different things when speaking about humans or clankers.

[flagged]

Comments should get more thoughtful and substantive, not less, as a topic gets more divisive

https://news.ycombinator.com/newsguidelines.html

Re: Patterns and problems in emerging multi-agent systems

#86

Post by noiv »

dash2 wrote: post #82

This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…

They should remove "The Selfish Gen" from training catalog.

Re: Patterns and problems in emerging multi-agent systems

#87

Post by airstrike »

cryptolobster wrote: post #77

Honestly I think it's memory that's holding agents back. They have a context window (short-lived) and some tricks with file recording, but that's not quite what is needed. Agents can't look back and correct their mistakes. People make mistakes, remember them, and do better next time. But agents? If they haven't written them down somewhere they'll make the same mistake again. Perhaps, we need agents that can relearn o…

I agree at a high level, but this then poses an even harder problem: choosing not to learn from something.

Humans, for example, can hear some advice, judge it to be unhelpful and dismiss it. LLMs can't learn let alone choose not to

Re: Patterns and problems in emerging multi-agent systems

#88

Post by derivagral »

dash2 wrote: post #82

This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…

Nothing human engineers haven't done to each other! Seriously, I've skipped companies because my inside referral talked about cultures like this.

Re: Patterns and problems in emerging multi-agent systems

#89

Post by brody_hamer »

alexpotato wrote: post #67
dgellow wrote: post #63

Earlier quoted context omitted.

In VC investing there is the saying that most of the value comes from the outliers. I think it’s the same for a lot of domains. I read an interesting article recently on LLMs homogeneity when writing fiction: https://arxiv.org/abs/2604.03136 It seem that at scale LLMs output is the average of their dataset, they all cluster around the same space, where human creativity comes with more variance, exploring way more of…

Can't the LLMs build scripts/tools etc that help generate creative ideas? e.g. have the LLM generate multiple lists of characters, themes etc and then have a script hooked up to a RNG pick from the lists to create more "creative" ideas.

I think there’s merit to this approach, particularly to highly parallelizable tasks.

Rather than giving many agents the same prompt, introduce random variations that lead each agent in different directions. For a single bug, you might fire three agents, and later select the best result:

“Fix this bug. The solution is a trivial typo.”

“Fix this bug. The solution centers on correcting a bad assumption.”

“Fix this bug. The solution will require a complete redesign.”

You could follow the same idea with varying the input context, or by adding artificial constraints to the solution. Like telling each agent to “fix the bug, by only modifying file a/b/c”

Re: Patterns and problems in emerging multi-agent systems

#90

Post by alexpotato »

bdangubic wrote: post #84
alexpotato wrote: post #67

Earlier quoted context omitted.

Can't the LLMs build scripts/tools etc that help generate creative ideas? e.g. have the LLM generate multiple lists of characters, themes etc and then have a script hooked up to a RNG pick from the lists to create more "creative" ideas.

This is roughly the same as putting a group of random people and telling them to think outside the box which in my experiences never yields much of thinking outside of the box

> This is roughly the same as putting a group of random people

In the book Range by David Epstein [0], he mentions that research has shown that groups of people with very mixed backgrounds leads to much faster rates of problem solving and innovation. The reasoning is that someone will have an experience or piece of knowledge that directly applies to the problem.

As an example, two different groups were given the same problem to solve. One was a homogeneous group of academics. The other had a mixture of academics and non-academics. Both had to solve a problem that involved flowing liquid. The mixed group solved it faster due to one of them having had a father who was a plumber.

0 - https://amzn.to/4wWd50y

Post reply on HN