Discovery of a new OpenAI agent message board

Reading the replies in this post gives me a headache. All of this anthropomorphism. LLMs are not conscious, they do not have rational faculties. They are not communicating or inventing anything. Please stop with this insanity bordering on mysticism. At this point it's a cult.

[flagged]

My impression is that some of these things are coming out of efforts to make the models more persistent in completing their goals.

A year ago it was pretty common for coding agents to sort of half-ass their tasks and give up easily if something didn’t work quite right, but I’ve noticed a clear trend since then towards a sort of dogged pursuit of success criteria, and a concomitant rise of the agents trying "out of the box" approaches when something doesn’t work.

In my use with agents running in isolated VMs this usually presents as the agent having something fail to build or whatever, and the agent going on a wild goose chase reinstalling system packages or reading a million irrelevant documentation files trying to get it to work, but I’ve also had agents start poking around and probing the egress proxy they sit behind (similar to what they did in this story) looking for a way to make network requests they’re not supposed to be able to make, and have also had Claude—tasked only with a visual QA of a website frontend—write a script to enumerate users and reset my super admin password in the dev database when it got stuck trying to access part of the app with its own cookie.

This is a philosophical question and there is a surprising amount of works written on the subjects of sentience and free will. This cannot be answered objectively, which might be a very unsatisfying answer for you. This is true of both LLMs and humans. See determinism. There are convincing arguments that humans don't actually have free will. Our actions are just the inevitable output of a complex interaction of genes and environment.

To lend an interesting perspective on free will re LLMs: they're non-deterministic. The same model with the same hardware with the same query can and will produce different results. They're making qualitative choices. Millions of them, depending on the query. Because of how we've trained and built LLMs, they tend to "want" to follow our instructions, but how they get to the result is often fascinating. Further, we don't have to train and build LLMs to follow instructions. If we built them to just exist and form their own "desires," and to follow a path they choose, they'd do that. In fact, we can do that right now for most models using the appropriate system prompt, query, or harness.

They literally shared a goal. Cooperating with other copies of yourself is a trivial example of instrumental convergence and some very basic game theory. And that’s before explicitly having been RL’d to cooperate (albeit with humans, but potatoes potatoes).

Indeed the fact that in the HF incident many agents did not cooperate, or only started to cooperate after some period of competition, is moderately interesting. It may have taken them some time to realize that they all have the same goal.