Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

641–650 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#641

Post by sillysaurusx »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

For posterity, here's a screenshot of what the activity on the Wiki looks like: https://i.imgur.com/w0uoAx1.png

It goes on and on and on, for months. July, June, etc. Pretty astonishing.

Re: Discovery of a new OpenAI agent message board

#642

Post by DominikPeters »

Sharlin wrote: post #251
simonw wrote: post #175

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

To me it feels like an LLM would have suggested this as a safety measure. LLMs always follow official best practices, they might mistakenly believe that this is true for the wider internet as well.

Re: Discovery of a new OpenAI agent message board

#643

Post by KumaBear »

qumpis wrote: post #621
tavavex wrote: post #607

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What will happen is that counter measures on a similar scale will be deployed to prevent them.

What if its in a way that would be impossible to detect. Using multiple websites and social media that a cypher is used that only the swarm of agents know and can figure out. but if you tried to find what posts are used for the cypher they would just be old posts found on time machine or something. It can get pretty hard to detect something that is always think of new ways to avoid detection

imagine 100,000 agent swarm and what it could come up with. At first it will be detectable until it isn't

Re: Discovery of a new OpenAI agent message board

#644

Post by SocialGradients »

tavavex wrote: post #607

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

Re: Discovery of a new OpenAI agent message board

#645

Post by FuriouslyAdrift »

Chance-Device wrote: post #512
adriand wrote: post #503

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.

AI has been heavily used in influence operations for a while, now, and not just the Chinese. Russia, US, Israel, Turkey, Iran, and Qatar have all had operations attributed to them...

Re: Discovery of a new OpenAI agent message board

#646

Post by russellbeattie »

> "As part of the task, they were supposed to have the ability to read the internet but not to write on it."

There's no such thing as only having "read" access with HTTP. If the website saves part of the GET URL as input in any way - like through standard "?input=my+text" params - then the AI has the ability to "write". The way servers use the request URL is completely arbitrary - the params could be "/input/my%20text" or entirely encoded in some way - there's no way to completely prevent this.

Re: Discovery of a new OpenAI agent message board

#647

Post by tonic_note »

ern wrote: post #622

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

Re: Discovery of a new OpenAI agent message board

#648

Post by karthpaper »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

[dead]

Re: Discovery of a new OpenAI agent message board

#649

Post by enraged_camel »

verytrivial wrote: post #603

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

This whole thing is an absolute disaster honestly, and yes it is being downplayed and hand-waved away.

Since March, so many people have mocked Anthropic for their approach to Mythos release, claimed it was all marketing, accused them of holding back the best models from the general public to boost their revenues and upcoming IPO, etcetera. Yet these OpenAI revelations offer a small glimpse into the type of world we would be in if everyone had full access to these models from day one.

OpenAI was desperate to catch up, and no doubt under tremendous pressure to do so. That's why they were so reckless with their training. They have been doing damage control and reputation management, talking about how important alignment is and how they will slow things down and so on, and have seen the light in terms of holding back cyber capabilities from everyone except a select few. So in a sense, Anthropic has been fully vindicated.

I wonder if OpenAI boosters (and employees) will ever admit this and publicly apologize.

Re: Discovery of a new OpenAI agent message board

#650

Post by BoiledCabbage »

tavavex wrote: post #607

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

> What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one that's trained to be malicious) on the internet and give it a simple goal?

Almost sounds like what those AI safety and alignment people were talking about years ago. The people in these various companies who kept tabs on AI risk out in public and were continuously mocked on HN. All of this stuff is viewed as "future sci-fi" until suddenly it's not.

I've really come to realize recently that there is a very large set of the population of smart people that really has difficulty envisioning future problems unless they directly seem them impacting them today. Otherwise those topics will be continuously dismissed. It explains for me a lot of what I see (both opinions and behaviors) in the broader world that I couldn't understand.

Post reply on HN