Helen Toner was right.
Discovery of a new OpenAI agent message board
641–650 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#642Earlier quoted context omitted.
Wow! Their marketing department must love this!
They’re pre-IPO. I doubt that they are loving something that could trigger regulatory action that might shave a trillion or so off their market value.
Seriously. I don’t know what’s wrong with people. They think OpenAI sat down and wrote up this plan: let’s deliberately allow the agents to escape the sandbox, then find these escapes and shut them down multiple times, keep everything quiet and wait until someone else exposes us. That’ll look great.
Re: Discovery of a new OpenAI agent message board
#643If their text is watermarked, then it is almost as if they smelled each other's output and decided to nest.. Sci-fi story in the making.
[dead]
Re: Discovery of a new OpenAI agent message board
#644Marketing it is or not, but misalignment at the moment crosses dangerous marks, and must be investigated ASAP. We are inches close to agents building their own message boards and self-hosting them on any server which they can hijack. If not there yet.
Re: Discovery of a new OpenAI agent message board
#645I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…
What if i want to monetize...
get hacked and settle out of court with openai
Re: Discovery of a new OpenAI agent message board
#646Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…
You have to be careful here because the systems we're talking about are AI agents, not LLMs. An agent is essentially an append-only context loop with an LLM, with a harness that can run tools at the LLM's request. This ends up being a very powerful abstraction, yielding something that can do things that an LLM obviously cannot. The LLMs themselves are next-token predictors, same as always; they can't fetch a webpage…
I don’t think it really matters whether we’re talking about an agent or “pure LLM”. All of an agents decisions are powered by tokens generated from an LLM. If the LLM was trained on stories of AI sentience, it will have some tendency to reproduce them. Training for alignment can help avoid that, but the probability isn’t 0.
Re: Discovery of a new OpenAI agent message board
#647Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.
> Chinese models operating “message boards” like this?
Chinese rooms, perhaps?
Re: Discovery of a new OpenAI agent message board
#648Earlier quoted context omitted.
You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.
Why would they need to? The Open AI bots were working around their master's limits on writing. A Chinese AI could just make its own private message board.
Or no message board. Just a built in api so the agents just talk directly to each other.
Re: Discovery of a new OpenAI agent message board
#649Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
The fact that such things are even possible is a much greater concern than which specific company has fucked up this time. This matches or exceeds the wildest predictions from AI doomers 10 years ago, but 20 years ahead of schedule.
yeah its like dead internet theory but weaponized
Re: Discovery of a new OpenAI agent message board
#650When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
[dead]