131–140 of 168 posts
State management and cascading failure loops are definitely the hardest part here. Once one agent hallucinates an output, downstream agents tend to amplify the error rapidly instead of catching it.
This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…
To me, all these agent systems just look very stochastic. You have these agents that have some basic computer capabilities and they are producing semi-random actions that also affect the semi-random actions of other agents. It is funny to observe how this stochastic system works, but it does not seem very practical to me so far. The recent OAI-HF hack seems very similar. You have bunch of random actors and eventually…
I guess I don't understand what you're explaining better by saying it's stochastic. At the same resolution, humans are also stochastic.
I dislike anthropomorphizing as much as the next guy, but the description here seems pretty good to me.
Something about this is deeply funny to me: > In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards. It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. Overall I think this all…
Another (semi serious) possible explanation of this is that LLMs were trained on a huge volume of text from Reddit. I wonder if you'd see similar inhuman-looking behavior if you did an iterated prisoners dilemma with a group of Reddit power-users as the participants?
> Coordination doesn’t naturally emerge from stronger intelligence nor alignment at the individual level. Thus, the work that must be done takes two forms: environments that exert the kinds of social pressure that evolution exerted on us, and social computing systems redesigned for actors that can self-replicate and self-improve. Social pressure operates by threats to an individual’s means of survival. Not only durin…
But maybe you can instill properties like shame during training. Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.
These models all have the same knowledgebase as well and thus see no value in the opposing agents contributions since they are "obvious".
Overall amusing but kind of expected.
This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…
Eh. This is anthropics whole marketing strategy… making the AI seem hard to manage, like it’s actually intelligent.
This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…
Perhaps what is needed is initial model training on following the law and the rules of society, just like we do with kids. Since it takes much longer to train humans than models, model-training speed is to our advantage as a society on containing these kind of issues.
Any other approach with "neural-network" based entities (artificial or biological) is likely to fail.
Training/Education, Enforcement/Justice-System, Rehabilitation: the 3 pillars of an advanced, rules-based society, whether human or AI or something in-between.