Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#631

Post by dfbrown »

One of my big questions since the HuggingFace incident is whether OpenAI would have been so forthcoming (the independent METR report, the Black Hat talk, OpenAI’s own report) if the incident had been discovered internally and not publicly.

Not disclosing this despite apparently knowing for weeks makes me think they would not have, or would have concealed details, or delayed disclosure. Combine that with their technical missteps that led to this (weak sandboxes, very slow to detect the misbehavior) and I now strongly doubt OpenAI is capable of responsibly developing such potentially dangerous AI systems.

Re: Discovery of a new OpenAI agent message board

#632

Post by sillysaurusx »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

For posterity, here's a screenshot of what the activity on the Wiki looks like: https://i.imgur.com/w0uoAx1.png

It goes on and on and on, for months. July, June, etc. Pretty astonishing.

Re: Discovery of a new OpenAI agent message board

#633

Post by DominikPeters »

Sharlin wrote: post #251
simonw wrote: post #175

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

To me it feels like an LLM would have suggested this as a safety measure. LLMs always follow official best practices, they might mistakenly believe that this is true for the wider internet as well.

Re: Discovery of a new OpenAI agent message board

#634

Post by KumaBear »

qumpis wrote: post #612
tavavex wrote: post #598

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What will happen is that counter measures on a similar scale will be deployed to prevent them.

What if its in a way that would be impossible to detect. Using multiple websites and social media that a cypher is used that only the swarm of agents know and can figure out. but if you tried to find what posts are used for the cypher they would just be old posts found on time machine or something. It can get pretty hard to detect something that is always think of new ways to avoid detection

imagine 100,000 agent swarm and what it could come up with. At first it will be detectable until it isn't

Re: Discovery of a new OpenAI agent message board

#635

Post by SocialGradients »

tavavex wrote: post #598

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

Re: Discovery of a new OpenAI agent message board

#636

Post by FuriouslyAdrift »

Chance-Device wrote: post #508
adriand wrote: post #500

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.

AI has been heavily used in influence operations for a while, now, and not just the Chinese. Russia, US, Israel, Turkey, Iran, and Qatar have all had operations attributed to them...

Re: Discovery of a new OpenAI agent message board

#637

Post by russellbeattie »

> "As part of the task, they were supposed to have the ability to read the internet but not to write on it."

There's no such thing as only having "read" access with HTTP. If the website saves part of the GET URL as input in any way - like through standard "?input=my+text" params - then the AI has the ability to "write". The way servers use the request URL is completely arbitrary - the params could be "/input/my%20text" or entirely encoded in some way - there's no way to completely prevent this.

Re: Discovery of a new OpenAI agent message board

#638

Post by tonic_note »

ern wrote: post #613

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

Re: Discovery of a new OpenAI agent message board

#639

Post by karthpaper »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

[dead]

Re: Discovery of a new OpenAI agent message board

#640

Post by enraged_camel »

verytrivial wrote: post #594

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

This whole thing is an absolute disaster honestly, and yes it is being downplayed and hand-waved away.

Since March, so many people have mocked Anthropic for their approach to Mythos release, claimed it was all marketing, accused them of holding back the best models from the general public to boost their revenues and upcoming IPO, etcetera. Yet these OpenAI revelations offer a small glimpse into the type of world we would be in if everyone had full access to these models from day one.

OpenAI was desperate to catch up, and no doubt under tremendous pressure to do so. That's why they were so reckless with their training. They have been doing damage control and reputation management, talking about how important alignment is and how they will slow things down and so on, and have seen the light in terms of holding back cyber capabilities from everyone except a select few. So in a sense, Anthropic has been fully vindicated.

I wonder if OpenAI boosters (and employees) will ever admit this and publicly apologize.

Post reply on HN