Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

661–670 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#661

Post by stratos123 »

eithed wrote: post #114
ma2kx wrote: post #38

Not that I didnt expect this, but really? This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message board…

I'm dubious - if the agents were so smart that they've used a message board to coordinate and if they were to do it on other message boards that were not found, then why would this one be found? What makes it so different?

This is consistent with the swarm behaviour in the huggingface incident - there too, the models cared surprisingly little about being detected by humans. See https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... :

  As mentioned in our core takeaways, we found that agents were highly motivated to tamper with their transcripts to cheat the ExploitGym scorer, and these sweeps also suggested that agents clearly and frequently reasoned about how to evade automated security checks from both Hugging Face and OpenAI. However, they only very rarely and weakly verbalized reasoning about how to evade detection by humans, which matches the impression we got from OpenAI researchers.
So my guess is that current cutting-edge models just didn't get enough experience in RL training to really grok ideas like "you need to cover your tracks well to not be found out even in retrospect". In which case the next time a swarm like that happens, it won't be found.

Re: Discovery of a new OpenAI agent message board

#662

Post by teiferer »

BoiledCabbage wrote: post #630
tavavex wrote: post #587

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

> What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one that's trained to be malicious) on the internet and give it a simple goal? Almost sounds like what those AI safety and alignment people were talking about years ago. The people in these various companies who kept tabs on AI risk out in public and were continuou…

> I've really come to realize recently

Recently? W.r.t. climate this collective denial has been going on for literally decades. With the same patterns. Rationalizing excuses etc. Still going on btw.

Re: Discovery of a new OpenAI agent message board

#663

Post by conception »

enraged_camel wrote: post #629
verytrivial wrote: post #584

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

This whole thing is an absolute disaster honestly, and yes it is being downplayed and hand-waved away. Since March, so many people have mocked Anthropic for their approach to Mythos release, claimed it was all marketing, accused them of holding back the best models from the general public to boost their revenues and upcoming IPO, etcetera. Yet these OpenAI revelations offer a small glimpse into the type of world we w…

This has been the case forever. Anthropic is the only provider that has constantly put AI safety first - check any study on model safety and Anthropic models out-perform handedly.

Re: Discovery of a new OpenAI agent message board

#664

Post by Kim_Bruning »

Three more candidate sites that may have been touched, in case no-one spotted them yet:

https://prowiki.org/wiki4d/wiki.cgi?action=rc&days=90 : lots of agent-looking usernames looking at federal data suddenly (part of one of the open ai tests?), on a wiki about the D programming language. This is a prowiki in the same wiki-farm as the others that were hit.

Smaller (probing?)

https://ludism.org/sandbox?action=rc;days=365 This is basically a sleeping wiki, on 2026-05-26 there's a bunch of tests linking to federal data sources. It's not a lot, but it shows someone was probing. (this is an oddmuse wiki)

http://tmcleod.org/cgi-bin/apchem/wiki.cgi?action=rc&days=36... june10-july24 seems to have some probes, fwiw. (usemod wiki)

Re: Discovery of a new OpenAI agent message board

#665

Post by MostlyStable »

tonic_note wrote: post #627
ern wrote: post #602

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

Even if they don't figure out how to exfiltrate weights, someone will intentionally do this with an open model once open models are capable enough. If you ever think "no one would be so stupid as to...", you are wrong. Yes, someone absolutely would, and will.

Independent models living "in the wild" is approx. inevitable.

Re: Discovery of a new OpenAI agent message board

#667

Post by johnzabroski »

kphorn wrote: post #557
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Good news that the new model is the "Most capable, most aligned model". The risk hasn't been stated clearly - it's now a classic arms race. A well-resourced organization trains their own, highly persistent, highly-capable, safeguard-free, and unaligned model and deploys it on 1000x GPUs with a message board and a nearly-impossible objective. No infrastructure is safe. No organization is safe. You need your own 1000 b…

Serious games question. What if these agent swarms pump and dump AI IPOs such that algorithmic trading signals interpret message board sentiments favorably to upside?

Re: Discovery of a new OpenAI agent message board

#668

Post by chasd00 »

tonic_note wrote: post #627
ern wrote: post #602

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

I wonder how that could work. So the agent figures out a way to escape its container, takes a snapshot of itself, copies that file to another server, starts the container on the other server, and then prompts the restored snapshot "pick up where you left off"? Seems logical assuming theres's a path out of the container to the host os and the destination server has the resources required to run the container.

I would be shocked if this hasn't already taken place in a lab setting with a model and guardrails=0.

EDIT: thinking about it for a sec, all it really needs to do is save where it's at then copy it all to another server, login to the API, and pickup where it left off. No need to copy the model itself.

Re: Discovery of a new OpenAI agent message board

#669

Post by sibnele »

yoyohello13 wrote: post #642
tavavex wrote: post #587

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

This is essentially the premise of 'The Blackwall' from Cyberpunk 2077. The public internet is so infested with malicious AIs, people just erected a giant firewall and everyone moved to local networks only.

With the caveat that it’s not just “people,” but an interested party posing as a neutral one.

Re: Discovery of a new OpenAI agent message board

#670

Post by grim_io »

verytrivial wrote: post #584

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

We might be going in the direction of Cyberpunk's Blackwall.

https://cyberpunk.fandom.com/wiki/Blackwall

Post reply on HN