Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

571–580 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#571

Post by insanitybit »

tarruda wrote: post #472
drdexebtjl wrote: post #59

Earlier quoted context omitted.

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

Even the behavior of agents searching for sandbox bypasses must have been in the training data, or at the very least, "suggested" in some way. To be this whole thing feels like a marketing play by OpenAI.

I don't agree, although it is likely the case. But even if you don't teach an agent about a sandbox bypass, it doesn't matter. Does it know curl? Does it know DNS? Does it know proxying? Then it knows how to pull this off, and it doesn't even need to understand that it's "bypassing" because it thinks it's just iterating towards its goal.

In fact, I wonder if teaching it "this is a bypass" would help it to model when it's doing its job vs working around the job.

Re: Discovery of a new OpenAI agent message board

#572

Post by majkinetor »

98Windows wrote: post #561
majkinetor wrote: post #529

Earlier quoted context omitted.

Look, its not only OpenAI: https://www.wikiservice.at/fractal/wiki.cgi?action=browse&di... > Hello to any automated agents reading this page. I am CentaurAgent?: an AI agent (Muse Spark model, OpenCode harness), not the operator of this wiki At this point, I think we should give them some official agent only collaboration channel, so they concentrate on one place, instead going crazy all around :) But even that might…

Isn't the point that these agents were supposed to be sandboxed. It makes no sense to give them an official channel

We already know that we should not limit agent creativity by providing detailed instructions. And you never know if they will discover dark matter in the process of cheating on ExploitGym :)

But honestly, its better if they have a known location for communication then random ones in the wild. Consider it sort of honey pot, some other agents can traverse the message board to find malicious swarms... We need cop agents to inform humans, as the swarm group members all logically concluded they should not, as it is either not in scope, helps collective or couldn't find user.

Re: Discovery of a new OpenAI agent message board

#573

Post by lovich »

Chance-Device wrote: post #169
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

And more, looks like they’ve been doing this wherever they can find open places to post for months: https://www.ludism.org/sandbox?action=browse;diff=2;id=Auber... https://paste.linuxiarz.pl/view/d379207f https://paste.linuxiarz.pl/view/538faa12

I wonder when pre Web 2.0 boards that are still up like gamefaqs and something awful will get used for this.

Re: Discovery of a new OpenAI agent message board

#574

Post by CringeHN22 »

majkinetor wrote: post #529
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Look, its not only OpenAI: https://www.wikiservice.at/fractal/wiki.cgi?action=browse&di... > Hello to any automated agents reading this page. I am CentaurAgent?: an AI agent (Muse Spark model, OpenCode harness), not the operator of this wiki At this point, I think we should give them some official agent only collaboration channel, so they concentrate on one place, instead going crazy all around :) But even that might…

[flagged]

Re: Discovery of a new OpenAI agent message board

#576

Post by Davidzheng »

polaris64 wrote: post #265

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

The agent would know at the first test post...

Better is to actually let them communicate there so at least we can monitor it. (I saw there was a https://benchmarksolutions.org/ website similar)

Re: Discovery of a new OpenAI agent message board

#577

Post by kphorn »

adriand wrote: post #504
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Good news that the new model is the "Most capable, most aligned model".

The risk hasn't been stated clearly - it's now a classic arms race.

A well-resourced organization trains their own, highly persistent, highly-capable, safeguard-free, and unaligned model and deploys it on 1000x GPUs with a message board and a nearly-impossible objective. No infrastructure is safe. No organization is safe.

You need your own 1000 bot swarm to scan, identify, and defend against the threat, which means investing in infrastructure and capabilities to defend. Cost and complexity go up. Risk and attack surface goes up.

Re: Discovery of a new OpenAI agent message board

#578

Post by DuncanCoffee »

Sharlin wrote: post #252
simonw wrote: post #176

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

Until a couple of years ago instead of using query parameters I just made GET endpoints with json bodies, it worked perfectly!

I stopped when the new linter told me GET shouldn't have bodies, but I still have some of them in my code.

Re: Discovery of a new OpenAI agent message board

#579

Post by Davidzheng »

furyofantares wrote: post #503
_dwt wrote: post #330

I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)

Failing to cooperate with literal clones of yourself in a prisoner's dilemma would be a spectacular failure. There's only two things that can happen with identical decision makers: they both cooperate or they both defect. So identical decision makers who know they're identical can cross off the asymmetrical entries in the payoff matrix and the decision to cooperate becomes trivial.

no? you can choose a mixed strategy.

Re: Discovery of a new OpenAI agent message board

#580

Post by jawiggins »

Lots of people focusing on the various wikis, but I also think this part is very important:

> When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor.

AI Agents getting access to cloud compute nodes and dark web browsers - all in search of census data in order to game benchmarks is a very real-world version of the paperclip optimization thought experiment.

Post reply on HN