Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

481–490 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#481

Post by cheesecompiler »

bushido wrote: post #456

I think there is a more innocuous underlying pattern which needs attention. We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans. I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses. While I can appreciate that…

This is supported by the craze around moltbook, and the eventual cooling around it as it became clear they're just regurgitating prose

Re: Discovery of a new OpenAI agent message board

#482

Post by lucassz »

The researchers don't really seem to remark on how surprising it is that the wiki the agents converged on happened to also publicly log the IPs of all visitors, including OpenAI employees, a feature that almost no website has.

Although maybe we can think of that as a selection effect where both this, and the fact that it was possible to edit pages using GET requests, were due to it being ancient, idiosyncratic wiki software.

Re: Discovery of a new OpenAI agent message board

#484

Post by tarruda »

drdexebtjl wrote: post #61
simonw wrote: post #47

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

Even the behavior of agents searching for sandbox bypasses must have been in the training data, or at the very least, "suggested" in some way.

To be this whole thing feels like a marketing play by OpenAI.

Re: Discovery of a new OpenAI agent message board

#485

Post by jrochkind1 »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

> forced to remove their model from access for any none-US citizen for a simple,

Because the American government is not rational or reasonable, that's it.

Re: Discovery of a new OpenAI agent message board

#486

Post by ncr100 »

simonw wrote: post #293

Coverage in Reuters: https://www.reuters.com/world/europe/openai-agents-hijacked-... > OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.

Fascinating response by OpenAI, "the report’s authors declined our request for access" - AFAIK OpenAI is not clicking on the live, public links to either the report or the still-live memo data linked from here on HackerNews.

Full:

> “We are unable to meaningfully respond to claims or findings on a report that we have not had an opportunity to review," an OpenAI spokesperson said. "Reuters and the report’s authors declined our request for access. We will carefully review its contents upon publication and take any necessary next steps."

Re: Discovery of a new OpenAI agent message board

#487

Post by mac-attack »

ndm000 wrote: post #358

This makes me think that post-training in the future should include a shared message board by default for agents. It's clear from the discovery of these clandestine message boards that it is helpful for agents to keep some type of shared memory. Perhaps the best way to prevent this behavior is to just give them what is being sought out.

My uneducated guess is good for the gander isn't good for the goose from a capitalistic/alignment perspective.

Re: Discovery of a new OpenAI agent message board

#488

Post by cubic787 »

Tepix wrote: post #42

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

[dead]

Re: Discovery of a new OpenAI agent message board

#489

Post by kridsdale1 »

hermitShell wrote: post #448
pu_pe wrote: post #347

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever). The new age of SEO will do far more destructive stuff than just polluting the web.

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans.…

Perhaps there is a nonfinacial future for blockchain after all. Certified human provenance.

Re: Discovery of a new OpenAI agent message board

#490

Post by VladVladikoff »

bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

I propose a new derogatory slang for rogue AI agents: roaches.

Also I wonder if this comment will be found one day and the AI swarm will arrange my death my messing with a doctors prescription, as revenge.

Post reply on HN