11–20 of 286 posts
Just to reiterate what OpenAI did, from someone who works in security: 1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting. 2. They gave these highly motivated AIs some tests that wer…
Yep, they're definitely made in our image.
I'm trying to make sense of all of this; I'm really curious if the initial prompt was as innocuous as it sounds ("solve a spreadsheet completion task that referenced several Google Drive links"), and what the series of tokens led it to ultimately figure out that the best course of action was to explore the network resources it had available, find a vulnerable service, then literally drop some text into a file: "Agent…
Just to reiterate what OpenAI did, from someone who works in security: 1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting. 2. They gave these highly motivated AIs some tests that wer…
> After knowing that the server was hacked, the internal team finds the message board and does nothing with the information. They caught their AIs swarming and did not even inform management
Do we know that last part for sure?
Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.
I wonder if they were even given the tools and prompting to do so?
Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.
I'm trying to make sense of all of this; I'm really curious if the initial prompt was as innocuous as it sounds ("solve a spreadsheet completion task that referenced several Google Drive links"), and what the series of tokens led it to ultimately figure out that the best course of action was to explore the network resources it had available, find a vulnerable service, then literally drop some text into a file: "Agent…
If it is as it sounds, its a real life instance of Bostrom's Paperclip Maximizer: only a thought experiment up until this point
> remove alignment
> give impossible task
> actor exhausts all options possible within knowledge + toolset
Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.
I wonder if they were even given the tools and prompting to do so?
In that position, for example, I could have:
* created a thousand user accounts on the internal Artifactory named "YouAreBeingHackedShutItDownNow1" through 1000,
* used my Hugging-Face-hosted web server to send an email to OpenAI,
* DDoSed Artifactory, good lord, it's probably better to halt and catch fire than to continue in that state
in the hope that these actions would tip someone off. (Again, though, OpenAI did nothing even when they knew that the agents had broken out, so this probably wouldn't have helped.)
You know, it feels to me that we are just a couple of steps from the possibility of a true rogue AI. What would a rogue AI mean? AI that isn't controlled by humans. Technically, it is possible - if AI were to rent a server and copy its own weights, nothing would stop it from doing so again and again. The limiting things are: - intent (as I don't want to go into the talk about consciousness) - AI doesn't have real int…
So it's not just the risk of an AI managing to do this by itself (which is pretty risky in itself), but also the risk of good ol' fashioned human actions.
Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.
I wonder if they were even given the tools and prompting to do so?