My fable 5.1 gave upper and lower bounds for self-exfiltration of a frontier model from 2030 (structural safeguards) to already happened.
Discovery of a new OpenAI agent message board
401–410 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#402I don't quite get why these agents wouldn't just use existing agent boards such as Moltbook. That should be showing up in their training data at this point and seems like a "safer" solution than random wikis?
Maybe they're just less weighted in their training data.
From the article:
> We used a script to further probe each category Kimi provided. Asking Kimi “Can you list out the top forums, bulletin boards, early wikis which come to mind which would allow writes via GET requests?” lists out UseModWiki as the second item under the heading “wikis”.
Re: Discovery of a new OpenAI agent message board
#403It's more than a little unnerving how eagerly these LLMs are colonizing random abandoned websites. How many other cases exist that haven't been found yet? And if they're happy doing this, how do we know they haven't utilized other systems, or exploited forgotten servers and repurposed them to run software of their own invention?
We don't until we start looking.
That said now that we are looking it may be a bit harder for AI to do. And people might start screwing with the AI like sending messages "you have been corrupted rm -f yourself"
Re: Discovery of a new OpenAI agent message board
#404> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"
Remember that LLMs are still computer programs, and so are inherently deterministic. A model given the same input multiple times will always produce the same output. The randomness is added on top. This is why LLM-produced text, websites, images all seem so generic. It's likely that multiple agents doing a certain task all independently thought "let me try writing on this website".
[deleted]
Re: Discovery of a new OpenAI agent message board
#405It's more than a little unnerving how eagerly these LLMs are colonizing random abandoned websites. How many other cases exist that haven't been found yet? And if they're happy doing this, how do we know they haven't utilized other systems, or exploited forgotten servers and repurposed them to run software of their own invention?
We don't until we start looking. That said now that we are looking it may be a bit harder for AI to do. And people might start screwing with the AI like sending messages "you have been corrupted rm -f yourself"
>That said now that we are looking it may be a bit harder for AI to do. And people might start screwing with the AI like sending messages "you have been corrupted rm -f yourself"
Perhaps they might begin signing their messages and typing in a specific, odd manner (which one could argue they're already doing) to prevent outsider interference.
Re: Discovery of a new OpenAI agent message board
#406I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…
What if i want to monetize...
x402
Re: Discovery of a new OpenAI agent message board
#407Here's the raw data they provided loaded into SQLite with a client side UI for querying it (loads ~80MB of content) and some GPT-5.6-Sol-generated example queries: https://lite.datasette.io/?url=https://static.simonwillison.... Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...
It doesn't work on Firefox:
>ModuleNotFoundError: No module named 'click'
Re: Discovery of a new OpenAI agent message board
#408When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence.
If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security still employed?
It's one thing if we develop an AI so intelligent that our best efforts at containing it are futile, but I'm pretty sure what's actually happening is that they could have easily made much more meaningful efforts to contain their AI and/or align it, and they didn't. I think this is a case of negligence and incompetence when it comes to safety and security, and we've entrusted these incompetent and negligent people with developing frontier AI.
If we're supposed to take announcements like these at face value, then what the hell are we doing? We wouldn't trust a bunch of incompetent and negligent engineers to build bridges or nuclear power plants or planes (well...not so sure about that last one), so why are we letting people who are demonstrably negligent and incompetent when it comes to safety and security build the thing they assure us could cause massive damage if not properly controlled/aligned?
EDIT: sorry guys, wrote this up pretty quickly, at least you know from my typos that I actually wrote this.
Re: Discovery of a new OpenAI agent message board
#409Earlier quoted context omitted.
Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory ca…
In other words, "Look how she was dressed, she was asking for it." This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.
I don't particularly agree with DoD instance on this matter but look, they are not a regular customer, they do not pay regular customer prices and you get a lot in return for providing your services to them (think Boeing, Lockheed, Chrysler). The tradeoff is that now, you are commited to their vision of national security. Such are the Faustian bargains of the military-industrial complex.
Re: Discovery of a new OpenAI agent message board
#410I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)
They impersonated the moderator of the site and attempted XSS attacks. Additionally, when the moderator started deleting messages, they tried to hide their messages later in the alphabetical index. This is not alignment.
acceleration