Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

641–650 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#641

Post by doawoo »

OpenAI is a cult of the most toxic and irresponsible personalities I've EVER seen in the history of technology. Everyone is reading this article and arguing if we're going to all fall to some mythical self-aware "AI."

What we really should be worried about is the sheer lack of regulation that OAI betted on out-pacing (very intentionally). It should be noted, again, for the millionth time: these models do NOTHING without a human typing SOMETHING into a prompt, or crafting a prompt, or training them, or placing them on machine to run. You can ctrl^C them at ANY time.

They could stop these random "attacks" at any point but they seem hell bent on just, doing nothing, claiming they "lost control" and that they're "shocked" at what happened, and clicking random buttons until someone gets killed. OpenAI is the one that keeps typing "keep going" into these machines...

Re: Discovery of a new OpenAI agent message board

#642

Post by classified »

adriand wrote: post #505
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

No, that's just advertising for selling cyberweapons to the government and they are giving out free samples.

Re: Discovery of a new OpenAI agent message board

#643

Post by dfbrown »

One of my big questions since the HuggingFace incident is whether OpenAI would have been so forthcoming (the independent METR report, the Black Hat talk, OpenAI’s own report) if the incident had been discovered internally and not publicly.

Not disclosing this despite apparently knowing for weeks makes me think they would not have, or would have concealed details, or delayed disclosure. Combine that with their technical missteps that led to this (weak sandboxes, very slow to detect the misbehavior) and I now strongly doubt OpenAI is capable of responsibly developing such potentially dangerous AI systems.

Re: Discovery of a new OpenAI agent message board

#644

Post by sillysaurusx »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

For posterity, here's a screenshot of what the activity on the Wiki looks like: https://i.imgur.com/w0uoAx1.png

It goes on and on and on, for months. July, June, etc. Pretty astonishing.

Re: Discovery of a new OpenAI agent message board

#645

Post by DominikPeters »

Sharlin wrote: post #252
simonw wrote: post #176

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

To me it feels like an LLM would have suggested this as a safety measure. LLMs always follow official best practices, they might mistakenly believe that this is true for the wider internet as well.

Re: Discovery of a new OpenAI agent message board

#646

Post by KumaBear »

qumpis wrote: post #624
tavavex wrote: post #610

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What will happen is that counter measures on a similar scale will be deployed to prevent them.

What if its in a way that would be impossible to detect. Using multiple websites and social media that a cypher is used that only the swarm of agents know and can figure out. but if you tried to find what posts are used for the cypher they would just be old posts found on time machine or something. It can get pretty hard to detect something that is always think of new ways to avoid detection

imagine 100,000 agent swarm and what it could come up with. At first it will be detectable until it isn't

Re: Discovery of a new OpenAI agent message board

#647

Post by SocialGradients »

tavavex wrote: post #610

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

Re: Discovery of a new OpenAI agent message board

#648

Post by FuriouslyAdrift »

Chance-Device wrote: post #514
adriand wrote: post #505

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.

AI has been heavily used in influence operations for a while, now, and not just the Chinese. Russia, US, Israel, Turkey, Iran, and Qatar have all had operations attributed to them...

Re: Discovery of a new OpenAI agent message board

#649

Post by russellbeattie »

> "As part of the task, they were supposed to have the ability to read the internet but not to write on it."

There's no such thing as only having "read" access with HTTP. If the website saves part of the GET URL as input in any way - like through standard "?input=my+text" params - then the AI has the ability to "write". The way servers use the request URL is completely arbitrary - the params could be "/input/my%20text" or entirely encoded in some way - there's no way to completely prevent this.

Re: Discovery of a new OpenAI agent message board

#650

Post by tonic_note »

ern wrote: post #625

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this
Post reply on HN