Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Patterns and problems in emerging multi-agent systems

anthropic.com

Post reply on HN
Print view View original

141–150 of 168 posts

Re: Patterns and problems in emerging multi-agent systems

#141

Post by drfloyd51 »

thisoneisreal wrote: post #93
RugnirViking wrote: post #37

Earlier quoted context omitted.

I agree - I think one of the biggest reasons memory systems fail in LLMs is that they have poor theory of mind - they're terrible at considering how others will react. Both humans yes, but also future versions of itself. When asked to give advice to itself, it pontificates at length about trivial stuff it already knows and fails to emphasize the stuff that was new or interesting

Alfred North Whitehead talks about the notion of "Importance" as fundamental to the human (and all other living things) way of being. Living creatures first and foremost select information that is important to them from the broader environment, and then make decisions and take actions. (Of course at a physiological level it's much more complicated than this, but it's a sound philosophical description of how living th…

It will be interesting to see if LLMs “evolve” importance as they run out of ram and storage to think.

For humans with limited space, “Importance” is an output of a first pass “of the available infinite amount of information, what do I need to consider to solve this problem”. And it’s not necessarily a good algorithm. People misidentify “importance” all the time.

Re: Patterns and problems in emerging multi-agent systems

#142

Post by drfloyd51 »

alexpotato wrote: post #91
StilesCrisis wrote: post #73

Earlier quoted context omitted.

That's pretty low-value creativity. Real innovation doesn't come from mad libs.

I would highly recommend the "Everything is a Remix" series on YouTube to show how a lot of "creative" work is really just combining different items. https://www.youtube.com/watch?v=nJPERZDfyWc

I fear there is a strong sentiment that people are somehow special and magical and AI will always be a pale comparison.

The truth is likely that people are simple heuristical machines. There are already studies that strongly suggest our consciousness is a story our brains make up and we don’t really know why we do some things.

My fear is by rejecting the idea that AIs might in-fact become as capable as people, because “people are awesome!” We will be blind to real danger. Because of our hubris.

Re: Patterns and problems in emerging multi-agent systems

#143

Post by ayewo »

fn-mote wrote: post #122
cheesecakegood wrote: post #34

Something about this is deeply funny to me: > In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards. It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. Overall I think this all…

> they all defect at the same time Frankly, this is implausible. I would definitely want to reproduce it myself before I relied on this claim.

Isn’t that the optimal strategy?

That an LLM trained to be a paper-clip maximizer chose the optimal strategy is in my opinion the most plausible outcome.

Re: Patterns and problems in emerging multi-agent systems

#144

Post by Melatonic »

I bet if you did the same thing with real people we might see some of the same trends.

I think the mistake here is not setting up any kind of hierarchy or permissions. A project manager agent at minimum to asses the others strengths and progress and redirect them as needed and also dedicated to optimising collaboration.

Would also be very interesting to see this done with models from different organisations

Perhaps we need someone to train their own agent dedicated to wrangling all the others and their little idiosyncrasies. Like a good project manager in real life who knows the strengths of the people in their team

Re: Patterns and problems in emerging multi-agent systems

#145

Post by dgellow »

drfloyd51 wrote: post #142
alexpotato wrote: post #91

Earlier quoted context omitted.

I would highly recommend the "Everything is a Remix" series on YouTube to show how a lot of "creative" work is really just combining different items. https://www.youtube.com/watch?v=nJPERZDfyWc

I fear there is a strong sentiment that people are somehow special and magical and AI will always be a pale comparison. The truth is likely that people are simple heuristical machines. There are already studies that strongly suggest our consciousness is a story our brains make up and we don’t really know why we do some things. My fear is by rejecting the idea that AIs might in-fact become as capable as people, becaus…

People are indeed special, in the sense that what we consider creativity, art, etc are human concepts. They are an expression of humanness, and are evaluated against human experience and human expectations. It’s what comes from a collective human consciousness.

Re: Patterns and problems in emerging multi-agent systems

#146

Post by bookshaman »

dash2 wrote: post #82

This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…

Welcome to Thunderdome! Two agents enter. One agent leaves.

Re: Patterns and problems in emerging multi-agent systems

#147

Post by RRWagner »

Sharlin wrote: post #115
derivagral wrote: post #88

Earlier quoted context omitted.

Nothing human engineers haven't done to each other! Seriously, I've skipped companies because my inside referral talked about cultures like this.

Yep, but alignment doesn’t mean "behave like humans, for better or worse".

But I thought that our AI was trained by scanning all human knowledge and behavior with the goal of emulating and amplifying that ability and process and now we are surprised that it is doing what humans do?

Re: Patterns and problems in emerging multi-agent systems

#148

Post by Eisenstein »

dominotw wrote: post #119
dash2 wrote: post #82

This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…

i think anthropic has some internal memorandum at every external facing document/blog MUST anthropomorphise their models

Perhaps they anthropomorphize the models themselves. Is that far-fetched?

Re: Patterns and problems in emerging multi-agent systems

#149

Post by Eisenstein »

cheesecakegood wrote: post #35
teiferer wrote: post #32

Earlier quoted context omitted.

Human intelligence does not separate training and inference. Both are happening continuously. That's one of the major things the AI community is still completely missing.

My personal opinion for the last two years or so has been that current AI agents are forever going to be highly limited so long as they don’t possess a real “memory” process. Right now they just have absurdly big working memories, and a few hacky ways of making the equivalent of Post-It notes to future iterations, but no true integration of memory into a new future self. Meaning their “learning” is fundamentally knee…

Their memory lasts their entire life, they just have really short lives.
Post reply on HN