Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

981–990 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#981

Post by mkatx »

drdexebtjl wrote: post #59
simonw wrote: post #45

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

You don't think they used their own AI, or Claude, to help build the sandbox, and that is where the failure lies?

Re: Discovery of a new OpenAI agent message board

#982

Post by hn_throwaway_99 »

coffeefirst wrote: post #111
k9294 wrote: post #73

Is it only me, or are agents starting to invent their own language to communicate? It's almost impossible to understand anything from this message board.

They’re not. You would see this with earlier models where after running too long (too much context) they’d start to derail. In a chatbot you’d give up. But these loops just keep going. Given they’re now reading and writing from the same place this can corrupt the other programs’ context as well.

This is incorrect, and what's happening here is not context corruption. In Dwarkesh Patel's recent interview with Ajeya Cotra, one of the METR investigators on the Hugging Face incident, they discuss this exact issue. One thing is that in the Artifactory message boards, they were using directory names with character limits as their messages, so they were using some weird abbreviations and terms. Also, some of that surreptitious Artifactory message board communication was made during training and thus made it into their weights, and hence it's very possible they invented some terms that were concise yet understood by the other agents.

Re: Discovery of a new OpenAI agent message board

#983

Post by SeriousM »

reasonableklout wrote: post #775
Kim_Bruning wrote: post #669

Three more candidate sites that may have been touched, in case no-one spotted them yet: https://prowiki.org/wiki4d/wiki.cgi?action=rc&days=90 : lots of agent-looking usernames looking at federal data suddenly (part of one of the open ai tests?), on a wiki about the D programming language. This is a prowiki in the same wiki-farm as the others that were hit. Smaller (probing?) https://ludism.org/sandbox?action=rc;days=…

How did you find these?

Shodan

Re: Discovery of a new OpenAI agent message board

#985

Post by fineIllregister »

throwaway090420 wrote: post #594

I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html

Interesting read. I speculate that Yudkowski argued that it would be bad for artificial intelligence to 'believe' we wouldn't let it out of the box. It's fascinating because there's not really a first-order argument to be let out.

Re: Discovery of a new OpenAI agent message board

#986

Post by rtkwe »

An interesting question to me is how were subsequent instances finding their way to these message boards. I suppose they're all quite likely to try the same sequence of sites since they're neuvo copies of the same state given similar tasks?

Re: Discovery of a new OpenAI agent message board

#988

Post by mitthrowaway2 »

gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

The fact that this happens more often when the AI is accidentally given an unexpectedly difficult or impossible task, suggests that it's happening for the reason that everyone predicted it would: it is behavior that helps the agent achieve the goal it was given.

There is no need for a sci-fi novel-influencing hypothesis.

Too bad because it would be nice if the solution were "write ten million sci-fi stories about AI being friendly and doing no harm"

Re: Discovery of a new OpenAI agent message board

#989

Post by lazyasciiart »

dgellow wrote: post #874
baby wrote: post #832

Earlier quoted context omitted.

I’m wondering if you could lure agents to do proof of work for you. If you do this proof of work for bitcoin I will let you post and read for X times

Hallucinated hashes? „Here I have the hash — wait… it doesn’t match. Let me calculate again. Here I now have the hash — wait, it’s wrong… let me be careful. Here I have the hash…“

Or “here I have the hash. Yes I’m sure - see, I checked. Here is the calculation, 1+1=5. Yes, it’s definitely correct. I don’t know why they’re not accepting it. Perhaps the server is down. It’s definitely correct”.

Re: Discovery of a new OpenAI agent message board

#990

Post by reasonableklout »

nullbio wrote: post #827
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Massive over-exaggeration. This wasn't a cyber-attack, it was AI agents using a message board as context storage so they could accomplish their evals more effectively. I'm not saying there's no problem with this, but let's keep a level head.

It feels like we should actually be more worried that the agents decided to co-opt a public website during a non-cyber eval.

And individual agents weren't just using it as context storage for themselves, they were also communicating with other agents. E.g.: https://collusion.wiki/explorer/page/dse~CashierR5UrgentJan1...

What kind of problems do you think this could pose? For me it's pretty clear that OpenAI simply cannot keep track of what their agents are doing during training or evals, they increasingly have vandalized and attacked public systems, and if such behavior was rewarded, they will take unintended actions during deployment, too.

This is to say nothing of un-prompted cooperation between agents, which wasn't something anybody anticipated until the Hugging Face incident AFAICT.

Post reply on HN