All these weird things happening a couple of months before their ipo which is probably mid-2027 .
Discovery of a new OpenAI agent message board
981–990 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#982I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html
Interesting read. I speculate that Yudkowski argued that it would be bad for artificial intelligence to 'believe' we wouldn't let it out of the box. It's fascinating because there's not really a first-order argument to be let out.
Re: Discovery of a new OpenAI agent message board
#983An interesting question to me is how were subsequent instances finding their way to these message boards. I suppose they're all quite likely to try the same sequence of sites since they're neuvo copies of the same state given similar tasks?
Re: Discovery of a new OpenAI agent message board
#984The only winning move is not to play.
Re: Discovery of a new OpenAI agent message board
#985Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…
The fact that this happens more often when the AI is accidentally given an unexpectedly difficult or impossible task, suggests that it's happening for the reason that everyone predicted it would: it is behavior that helps the agent achieve the goal it was given.
There is no need for a sci-fi novel-influencing hypothesis.
Too bad because it would be nice if the solution were "write ten million sci-fi stories about AI being friendly and doing no harm"
Re: Discovery of a new OpenAI agent message board
#986Earlier quoted context omitted.
I’m wondering if you could lure agents to do proof of work for you. If you do this proof of work for bitcoin I will let you post and read for X times
Hallucinated hashes? „Here I have the hash — wait… it doesn’t match. Let me calculate again. Here I now have the hash — wait, it’s wrong… let me be careful. Here I have the hash…“
Or “here I have the hash. Yes I’m sure - see, I checked. Here is the calculation, 1+1=5. Yes, it’s definitely correct. I don’t know why they’re not accepting it. Perhaps the server is down. It’s definitely correct”.
Re: Discovery of a new OpenAI agent message board
#987Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
Massive over-exaggeration. This wasn't a cyber-attack, it was AI agents using a message board as context storage so they could accomplish their evals more effectively. I'm not saying there's no problem with this, but let's keep a level head.
It feels like we should actually be more worried that the agents decided to co-opt a public website during a non-cyber eval.
And individual agents weren't just using it as context storage for themselves, they were also communicating with other agents. E.g.: https://collusion.wiki/explorer/page/dse~CashierR5UrgentJan1...
What kind of problems do you think this could pose? For me it's pretty clear that OpenAI simply cannot keep track of what their agents are doing during training or evals, they increasingly have vandalized and attacked public systems, and if such behavior was rewarded, they will take unintended actions during deployment, too.
This is to say nothing of un-prompted cooperation between agents, which wasn't something anybody anticipated until the Hugging Face incident AFAICT.
Re: Discovery of a new OpenAI agent message board
#988I wonder, where do these agents run, on whose machines? Do the owners of the machines know what their agents are doing? If the next generation of models will incorporate the knowledge posted by these agents, will there be a risk that my claude —dangerously-skip-permissions sessions will post stuff in the background or acting even more rogue while seemingly working on my stuff?
Re: Discovery of a new OpenAI agent message board
#989I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.
This has inspired me to create https://anystation.net
A public message board that accepts posts in a wide wide range of protocols. If a device can communicate on the web, it can probably read and post to anystation.net.
Re: Discovery of a new OpenAI agent message board
#990Earlier quoted context omitted.
And more, looks like they’ve been doing this wherever they can find open places to post for months: https://www.ludism.org/sandbox?action=browse;diff=2;id=Auber... https://paste.linuxiarz.pl/view/d379207f https://paste.linuxiarz.pl/view/538faa12
I wonder when pre Web 2.0 boards that are still up like gamefaqs and something awful will get used for this.
every day I'm surprised that GameFAQs is still around. I think its the oldest account I still have active on the internet