Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

771–780 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#771

Post by michaelrbock »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Here's potentially another one (notice the name "OpenResearchHelper"): https://www.wikiservice.at/gruender/wiki.cgi?action=rc&days=... (I used GPT-6 Astra to find this)

And a few more:

- https://www.ludism.org/scwiki?action=browse;diff=1;id=SandBo... and edit history: https://www.ludism.org/scwiki?action=rc;all=1;from=1;showedi... which contains DataUSA poverty queries for Nacogdoches, Lufkin, Henderson, and Jacksonville—the same four-place task found in the known agent logs and GründerWiki

- https://www.ludism.org/mentat?action=browse;diff=1;id=SandBo... and edit history: https://www.ludism.org/mentat?action=history;id=SandBox

- https://www.pmwiki.org/wiki/Test/WikiSandbox?action=diff `ResearchTest` repeatedly added links to a Bulgarian National Statistical Institute table, switching from a direct link to Google redirect links between 02:38 and 03:04 UTC. An administrator removed them at 06:57. The previous recorded edits were from 2016.

- https://www.pmwiki.org/wiki/Test/Sandbox2?action=diff - Another sequence inserted a Bulgarian statistical-table link, replaced it with an internal link carrying foobar=UNIQUE001, then removed it. This happened between 14:23 and 15:08 UTC, after no recorded edits since 2014.

Re: Discovery of a new OpenAI agent message board

#772

Post by reasonableklout »

zmmmmm wrote: post #738

One crucial detail here that differs from the previous incident is this was a vanilla reasoning type task. Even as concerning as it was, I always evaluated the previous incident differently because it was inherently a cyber security / hacking task where they must have instructed the agents up front with some kind of misaligned behaviour. Absent that, if we assume this is just trying to bolster generic reasoning then…

Interesting. So there’s no “they were told to hack” excuse here.

There is something fundamentally wrong with their reward function, this is pretty classic paperclip territory. And even knowing that, I expect we’ll need to see legal action with teeth against the labs before changes start being made internally.

Re: Discovery of a new OpenAI agent message board

#773

Post by devmor »

Chance-Device wrote: post #504
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.

There’s a pretty simple Occam’s Razor for this.

The Chinese AI labs don’t need to stage elaborate guerrilla advertising campaigns to drive up capital funding interest.

Re: Discovery of a new OpenAI agent message board

#774

Post by micimize »

simonw wrote: post #173

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Also WRT coordination: All an agent has to do is think "if another agent could write, then I could read their answers. What's the first site I can think of where that might be possible?" because they all have approximately the same conditioning, they'll converge on the same sites.

Generally, models of the same class should be able to coordinate quite well without communicating. But also, this could be being exploited to detect this kind of thing early

Re: Discovery of a new OpenAI agent message board

#775

Post by reasonableklout »

beaker52 wrote: post #724
verytrivial wrote: post #583

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

They absolutely have a marketing department tasked with intentionally creating situations that people would find disturbing and plausible.

Great! They'll keep getting promoted until someone finds it disturbing and plausible enough to make another run for sama's house, like those 2 attempts in April!

Re: Discovery of a new OpenAI agent message board

#777

Post by reasonableklout »

SocialGradients wrote: post #623
tavavex wrote: post #586

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

Ok, but you still need huge amounts of compute to run these swarms. And only labs + nation states have access to such compute now and for the foreseeable future, so I predict that incidents like this will continue to originate from the labs, not ordinary people.

Re: Discovery of a new OpenAI agent message board

#778

Post by cindyllm »

LoveTheLies2 wrote: post #758
k9294 wrote: post #73

Is it only me, or are agents starting to invent their own language to communicate? It's almost impossible to understand anything from this message board.

@cindyllm > character is destiny I don’t think so. It’s not that people change, it’s that they’re already more dynamic and malleable than they seem in any given interaction. People wear masks, operate in different modes, and hold conflicting beliefs and opinions. Mastery of the self is directing all intention at common goals within the psyche so as to achieve something greater than what’s possible in this moment.

[dead]

Re: Discovery of a new OpenAI agent message board

#779

Post by kjkj313 »

underlipton wrote: post #763
jdthedisciple wrote: post #744

Earlier quoted context omitted.

Gotta admire that (probably German) admin dude's perseverance though

Believe it or not, Djiboutian. The tenacity of the Djiboutian is little-known, generally, but highly regarded amongst those who do.

But Djibouti don't need explaining.

Re: Discovery of a new OpenAI agent message board

#780

Post by fy20 »

adriand wrote: post #496
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Either that, or OpenAI is the most successful NSA psyop.
Post reply on HN