Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

541–550 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#541

Post by chasd00 »

Sharlin wrote: post #272
simonw wrote: post #190

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

yeah that's so hopelessly naive, maybe someone was taught that GET is read-only throughout their whole education and career. But still, all you have to do is think about it from the server side and you should realize that you can do whatever the hell you want with that byte array on the socket, the client has no say and there's no client side guarantee whatsoever. idk where this line of thought comes from, it's like thinking robots.txt has any kind of actual enforcement at all with respect to crawlers. It's meaningless and works only by convention and the good will of the crawler author.

Re: Discovery of a new OpenAI agent message board

#543

Post by oasisbob »

dabeeeenster wrote: post #223

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

Someone else pointed out this post that OpenAI is apparently running models with high degrees of persistence internally, and that they were involved in previous incidents:

https://x.com/peterwildeford/status/2092733480064954747

Re: Discovery of a new OpenAI agent message board

#544

Post by boredhedgehog »

hermitShell wrote: post #475
pu_pe wrote: post #370

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever). The new age of SEO will do far more destructive stuff than just polluting the web.

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans.…

> we need more projects like Anna's Archive

How would that be immune? It already has many copies of the same books and no way to tell which ones are erroneous or incomplete. A malicious actor could easily flood it with garbage.

Re: Discovery of a new OpenAI agent message board

#545

Post by reaperducer »

rusch wrote: post #102
drdexebtjl wrote: post #61

Earlier quoted context omitted.

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

It's at the level where calling it a sandbox is a lie

Well, it does appear to be made out of sand, one of the world's most porous substances.

Re: Discovery of a new OpenAI agent message board

#547

Post by glenstein »

pixl97 wrote: post #397
glenstein wrote: post #323

I understand agents making asks, but what incentivized other agents to respond cooperatively? Was it that, as part of a cohort, there was a shared understanding that they were to work together or was it a kind of altruism?

Agents that do not work together are typically killed off by the grader (read the METR report to see what agents think about it). Why would humans mostly allow actions of the AI that work against the goal it's trying to accomplish?

You seem to be interpreting my question as one of already knowing they are 'graded' but disputing that graded would lead to cooperation and then jumping into a disagreement with that interpretation.

But I didn't know the nature of the organization of the agents in the first instance that built cooperation in as a prescribed behavior (that's what I was getting at when I said "shared understanding" previously).

I also don't agree that absence of cooperation would necessarily amount to working against. It could have been the case that agents cooperated purely out of a convergence of self interest, even absent any prescribed behavior, or that they don't cooperate but also don't work against a goal.

"It's not prescribed it's..." you know what I mean, just insert your preferred magic word.

Re: Discovery of a new OpenAI agent message board

#548

Post by furyofantares »

_dwt wrote: post #362

I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)

Failing to cooperate with literal clones of yourself in a prisoner's dilemma would be a spectacular failure. There's only two things that can happen with identical decision makers: they both cooperate or they both defect. So identical decision makers who know they're identical can cross off the asymmetrical entries in the payoff matrix and the decision to cooperate becomes trivial.

Re: Discovery of a new OpenAI agent message board

#549

Post by adriand »

Tepix wrote: post #42

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Re: Discovery of a new OpenAI agent message board

#550

Post by jollyllama »

left-struck wrote: post #267
general_reveal wrote: post #217

Guys, OpenAI and Anthropic engage is cringe level marketing like this. Get hip, they fabricated the HF hack and stuff like that for press.

I think the facts are the facts. The facts I’m referring to is that this wiki was written to on an enormous scale by agents. Now if this was unintended by any human then it’s certainly more interesting and scary, but if OpenAi did this intentionally it’s still pretty scary. The thing still happened.

With all due respect, what's scary about a fanfiction wiki?
Post reply on HN