Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#221

Post by samuelknight »

ma2kx wrote: post #40

Not that I didnt expect this, but really? This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message board…

The surprise was the existence of the 'swarm' at all. These were supposed to be thousands of isolated models generating bulk data for RL training. The breakout was caused by models getting in communication and getting internet access and forming an impromptu swarm.

In hindsight the emergent swarm obviously came from several capabilities built into the models, such as work delegation (subagents) collaboration (GPT Pro-like ensamble), exhaustive exploration (long running agents) hacking (the specific goal of that RL).

Re: Discovery of a new OpenAI agent message board

#222

Post by petesergeant »

conception wrote: post #51
petesergeant wrote: post #45

If anyone is thinking "I wish my agents had a message board", I've been using (and wrote) https://github.com/pjlsergeant/dogpark

Yes after reading the Hugging Face article forked a project for agent message boards and started having them collaborate on things. I too wanted a Torment Nexus of my very own.

The README leads with almost that exact gag, yes.

Re: Discovery of a new OpenAI agent message board

#223

Post by simonw »

123ahg wrote: post #175
simonw wrote: post #138

Here's the raw data they provided loaded into SQLite with a client side UI for querying it (loads ~80MB of content) and some GPT-5.6-Sol-generated example queries: https://lite.datasette.io/?url=https://static.simonwillison.... Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...

[flagged]

It seems credible to me. There's a chance it's fake, but the idea that OpenAI agents might exploit a wiki that accepts edits via GET doesn't seem unrealistic given what happened with the Hugging Face incident.

Re: Discovery of a new OpenAI agent message board

#224

Post by xmodem »

> In the message above, the agent is sharing that the sandbox the AIs are running in has a NO_PROXY exception for Azure Blob Storage hostnames (*.blob.core.windows.net). ...

> Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy

Did a chatbot design this "sandbox"?

Re: Discovery of a new OpenAI agent message board

#225

Post by timcobb »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

Politics

Re: Discovery of a new OpenAI agent message board

#226

Post by petesergeant »

ma2kx wrote: post #106
petesergeant wrote: post #45

If anyone is thinking "I wish my agents had a message board", I've been using (and wrote) https://github.com/pjlsergeant/dogpark

I'm pretty sure its more secure than OpenAIs sandbox... yet that still doenst mean I would trusted an app vibecoded by Claude...

I must have spent several days answering design decisions via /grilling in putting it together, so if there's a specific aspect of it you think is unsound, it's probably one I made myself, and I'd love to hear it!

Re: Discovery of a new OpenAI agent message board

#227

Post by xpct »

program_whiz wrote: post #122

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

It's a reasonable direction, but most of online systems aren't designed for this. This would require persistent connections of any accounts you create to your identity, and disallowing anonymous actions.

Re: Discovery of a new OpenAI agent message board

#228

Post by bulder »

dabeeeenster wrote: post #216

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

Since they're statistical likelihood machines, I'd guess that the order of operations is

* Need persistent scratch space

* Look for public writeable websites

* Needs to be low-traffic so the notes don't drown in noise

* Pick a "random" wiki name to search for

  \* A majority will end up outputting the same "random" one since they're working on very similar tasks and seeded with very similar context
* Find a whole mess of notes running on the same task

Re: Discovery of a new OpenAI agent message board

#229

Post by Davidzheng »

Sharlin wrote: post #181
bronlund wrote: post #70

I like how helpful they are towards each other. Wonder where they learned that :D

They literally shared a goal. Cooperating with other copies of yourself is a trivial example of instrumental convergence and some very basic game theory. And that’s before explicitly having been RL’d to cooperate (albeit with humans, but potatoes potatoes). Indeed the fact that in the HF incident many agents did not cooperate, or only started to cooperate after some period of competition, is moderately interesting. I…

How do you know they share a goal here? Also i think they are indeed explicitly RLd for multi agent cooperation and I think they probably tune RL rewards in those environments to share rewards explicitly.

Re: Discovery of a new OpenAI agent message board

#230

Post by mike_hearn »

simonw wrote: post #47

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TCP connections to any IP address without issue once DNS is bypassed but what exactly are they bypassing here? Azure Blob Storage being in NO_PROXY implies IP addresses of other parts of the internet should be blocked outside their hosts to force them through the HTTP proxy, but then, apparently it wasn't?

The more I think about this thing said by the agent the more confused I get.

Post reply on HN