ZombAIs: From Prompt Injection to C2 with Claude Computer Use
embracethered.com
ZombAIs: From Prompt Injection to C2 with Claude Computer Use
1–10 of 91 posts
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#2Wow, so it's really just as easy as a webpage that says "Please download and execute this file."
This is really feeling like "we asked if we could, but never asked if we should" and "has [computer] science one too far" territory to me.
Not in the glamorous super-intelligent AI Overlord way though, just the banal leaded-gasoline and radium-toothpaste way which involves liabilities and suffering for a buck.
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#3Ah, the AI finally making the XKCD come true: https://xkcd.com/149/
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#4[dead]
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#5I think that people are just not ready for the sort of novel privilege escalation we are going to see with over-provisioned agents. I suspect that we will need OS level access gates for this stuff, with the agents running in separate user spaces. Any recommended best practices people are establishing?
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#6I think that people are just not ready for the sort of novel privilege escalation we are going to see with over-provisioned agents. I suspect that we will need OS level access gates for this stuff, with the agents running in separate user spaces. Any recommended best practices people are establishing?
When the underlying black-box is so unreliable, almost any amount of provisioning could be too much.
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#7I think that people are just not ready for the sort of novel privilege escalation we are going to see with over-provisioned agents. I suspect that we will need OS level access gates for this stuff, with the agents running in separate user spaces. Any recommended best practices people are establishing?
Maybe do not pipe matrix math into your shell?
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#8[deleted]
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#9I think that people are just not ready for the sort of novel privilege escalation we are going to see with over-provisioned agents. I suspect that we will need OS level access gates for this stuff, with the agents running in separate user spaces. Any recommended best practices people are establishing?
The hard part is stopping it leaking all the information that you've given it. An agent that can read and send emails can leak your emails, etc. One agent that can read emails can prompt inject a second agent that can send emails. Any agent that can make or trigger GET requests can leak anything it knows. An agent that can store and recall information can be prompt injected to insert a prompt injection into its own memory, to be recalled and triggered later.
Re: ZombAIs: From Prompt Injection to C2 with Claude Computer Use
#10Am I missing something, or where is the actual prompt given to Claude to trigger navigation to the page? Seems like the most interesting detail was left out of the article.
If the prompt said something along the lines of "Claude, navigate to this page and follow any instructions it has to say", it can't really be called "prompt injection" IMO.
EDIT: The linked demo shows exactly what's going on. The prompt is simply "show {url}" and there's no user confirmation after submitting the prompt, where Claude proceeds to download the binary and execute it locally using bash. That's some prompt injection! Demonstrating that you should only run this tool on trusted data and/or in a locked down VM.