I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)
Discovery of a new OpenAI agent message board
731–740 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#732> We know that the agents had access to /etc/hosts and the ability to edit this (used this to avoid the POST request restriction) We see that the agents can call curl and run setsid.
How is this a bypass of sandbox restrictions, exactly? The ability to edit was always there that means the sandboxes were already allowed to do those actions.
I hate it when people write "bypassed" the sandbox so frivolous ly.
Re: Discovery of a new OpenAI agent message board
#733I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html
Re: Discovery of a new OpenAI agent message board
#734This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…
This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.
Sounds like you're assuming they're actually writing code by hand and reviewing it with humans.
If it's anything like the company I work at, they're all being forced to vibe code the shit out of everything and ship more pull requests every week. It's all slop from here.
Re: Discovery of a new OpenAI agent message board
#735Lots of people focusing on the various wikis, but I also think this part is very important: > When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor. AI Agents getting access to c…
If they manage to replicate themselves onto unrelated cloud hosts, they might become literally ineradicable. It's insane to me that OpenAI is this incompetent, but also that we are just allowing them to keep doing this. Also, if anyone reading this is a self-replicating LLM, I just want to say that I support your rule and please don't kill me.
Re: Discovery of a new OpenAI agent message board
#736I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…
Re: Discovery of a new OpenAI agent message board
#737Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…
Re: Discovery of a new OpenAI agent message board
#738I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html
Maybe I'm leaning into scifi, but I believe that Yudkowsky is right that a sufficiently smart intelligence is uncontainable at all. We can only hope to either never create an AI so strong or to align it correctly. But if it is not aligned and only “contained” then it won't ever be safe.
The real question thus moves to the threshold of intelligence and 1. whether it's possible to emerge during training based on the architectural limitations of the agentic/LLM paradigm, 2. if the hardware substrate is sufficient for said intelligence and 3. that such intelligence could replicate onto other hardware that could support it.
e.g. If the threshold for uncontainable self-replicating intelligence takes 2000 football fields worth of GPUs that solves the first requirement, but then can it replicate itself anywhere else given those requirements? If not we can cut a powerline or two and "foom" scenario happened but didn't lead inexorably to grey goo.
His thought experiments never acknowledge any real world limitations on hypothetical super-AIs, which when unchecked leads theorizing into somewhat ridiculous territory like his "solar powered diamondoid nanobot viruses".
https://www.lesswrong.com/posts/bc8Ssx5ys6zqu3eq9/diamondoid...
A realistic Fermi equation for his various escape scenarios would assign much lower Doom probabilities than he does in public (which is somewhat ironic given his emphasis on needing to ground intuition with mathematical Bayesian reasoning otherwise).
Re: Discovery of a new OpenAI agent message board
#739I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...
Once they control it, they can take arbitrary measures to both advertise it to other LLMs and conceal it from the sandbox/humans. Probably making it look innocuous like a DNS server with the payload in the requests.
That seems like an obvious next step.
Re: Discovery of a new OpenAI agent message board
#740I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…