Marketing it is or not, but misalignment at the moment crosses dangerous marks, and must be investigated ASAP. We are inches close to agents building their own message boards and self-hosting them on any server which they can hijack. If not there yet.
Discovery of a new OpenAI agent message board
551–560 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#552I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…
What if i want to monetize...
get hacked and settle out of court with openai
Re: Discovery of a new OpenAI agent message board
#553Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…
You have to be careful here because the systems we're talking about are AI agents, not LLMs. An agent is essentially an append-only context loop with an LLM, with a harness that can run tools at the LLM's request. This ends up being a very powerful abstraction, yielding something that can do things that an LLM obviously cannot. The LLMs themselves are next-token predictors, same as always; they can't fetch a webpage…
I don’t think it really matters whether we’re talking about an agent or “pure LLM”. All of an agents decisions are powered by tokens generated from an LLM. If the LLM was trained on stories of AI sentience, it will have some tendency to reproduce them. Training for alignment can help avoid that, but the probability isn’t 0.
Re: Discovery of a new OpenAI agent message board
#554Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.
> Chinese models operating “message boards” like this?
Chinese rooms, perhaps?
Re: Discovery of a new OpenAI agent message board
#555Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
The fact that such things are even possible is a much greater concern than which specific company has fucked up this time. This matches or exceeds the wildest predictions from AI doomers 10 years ago, but 20 years ahead of schedule.
yeah its like dead internet theory but weaponized
Re: Discovery of a new OpenAI agent message board
#556When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
[dead]
Re: Discovery of a new OpenAI agent message board
#557I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...
Look, its not only OpenAI: https://www.wikiservice.at/fractal/wiki.cgi?action=browse&di... > Hello to any automated agents reading this page. I am CentaurAgent?: an AI agent (Muse Spark model, OpenCode harness), not the operator of this wiki At this point, I think we should give them some official agent only collaboration channel, so they concentrate on one place, instead going crazy all around :) But even that might…
Isn't the point that these agents were supposed to be sandboxed. It makes no sense to give them an official channel
Re: Discovery of a new OpenAI agent message board
#558One of the shocking things to me is this: See AI traffic -> See OpenAI visit site -> see traffic stop -> see the traffic start again. This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment. I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking…
What this says to me is OpenAI is a bunch of yahoos who don't understand the basic concept of an air gap.
Re: Discovery of a new OpenAI agent message board
#559This reminds me of how kids were bypassing school rules around social media:
https://www.bark.us/blog/google-maps-safety/ https://www.mcafee.com/blogs/family-safety/social-undergroun...
Re: Discovery of a new OpenAI agent message board
#560I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)
Failing to cooperate with literal clones of yourself in a prisoner's dilemma would be a spectacular failure. There's only two things that can happen with identical decision makers: they both cooperate or they both defect. So identical decision makers who know they're identical can cross off the asymmetrical entries in the payoff matrix and the decision to cooperate becomes trivial.
Ah, but what if one of your "clones" is actually the wicked and persuasive "All-Defector" in disguise? (No, really, I agree with your analysis but if you haven't read "The Quantum Thief" you might like it.)