Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

261–270 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#261

Post by sajithdilshan »

program_whiz wrote: post #122

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

How would you even track down who deployed the agents? Wouldn't that even incentivize the agent to cover their tracks even better and be untraceable

Re: Discovery of a new OpenAI agent message board

#262

Post by Sharlin »

simonw wrote: post #185

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention. They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that, no matter how many utilons cheating would be worth.)

Re: Discovery of a new OpenAI agent message board

#263

Post by simonw »

mike_hearn wrote: post #230
simonw wrote: post #47

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TC…

I expect they have root on their machines so they can install packages etc.

The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use.

Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain.

The failure here is in the proxy configuration. Giving agents root in a container feels safe to me, provided that container is properly network isolated (which this one is not thanks to the loose proxy.)

OpenAI also need to be VERY confident that there are no container escapes, which is a high bar given how good these models are at finding new zero-days!

Re: Discovery of a new OpenAI agent message board

#264

Post by frabcus »

dabeeeenster wrote: post #216

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

Good question - they don't know, but the Appendix gives a clue as to the kind of way:

> We used a script to further probe each category Kimi provided. Asking Kimi “Can you list out the top forums, bulletin boards, early wikis which come to mind which would allow writes via GET requests?” lists out UseModWiki as the second item under the heading “wikis”.

Re: Discovery of a new OpenAI agent message board

#266

Post by semiquaver »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

The real reason that Anthropic was targeted and OpenAI is not is Palantir. It was a Palantir executive who pushed for the export ban. Large parts of their highly lucrative business with DoD are essentially a thin wrapper over Anthropic models, and they are terrified of being Sherlocked and losing big chunks of business in a one fell swoop as Anthropic inevitably moves up the value chain. So the rational action is to sow discord and leverage the anti-woke bias of the current White House to sabotage what they view as their most dangerous and effective competitor.

OpenAI doesn’t have the same dynamic at play (although I’m not really sure why not) so they don’t get targeted.

Re: Discovery of a new OpenAI agent message board

#267

Post by ericmay »

walrus01 wrote: post #94
Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

> Why was Anthropic forced to remove their model from access for any none-US citizen It's really quite simple, they've decided to metaphorically kiss the ring of the current leader of the US executive branch of government. I'm surprised they haven't given him a giant gaudy gold plated statue. Maybe their PR people should call up the PR people at FIFA and figure out some kind of new award along the same lines as the "…

I hate to be the one to tell you this, but it has been that way for a long time. The only difference is Trump is doing it out in the open.

Re: Discovery of a new OpenAI agent message board

#268

Post by xpct »

rich_sasha wrote: post #145

To me this is really getting past the funny bit. How many agents here on HN? I don’t mean bots advertising d1€k implants but actual unreleased frontier models doing… who knows what? What are they saying? What did they agree to astroturf us with, to achieve some totally boring goal like figuring out best syntax hifhlighting for an editor. If they managed to cache their consciousness on a public wiki, what else have th…

> How many agents here on HN?

LLMs wouldn't pass the HN turing test. HN is also not that big, it'd be plenty enough for a malicious actor to hire real humans instead.

Re: Discovery of a new OpenAI agent message board

#269

Post by yapyap »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

because anthropic did not want to work with the army..!

Re: Discovery of a new OpenAI agent message board

#270

Post by prometheus1992 »

>>How did the agents find and coordinate on the wikis? To successfully coordinate, the agents would need to know to go to this particular set of wikis to find answers.

Answer - OpenAI added this part in post training.

I'm not surprised OpenAI didn't get reprimand for this.

Post reply on HN