Given that the inference engine is dealing with untrusted inputs by definition, presumably you would want to sandbox it anyway. I don't think it matters whether it's the inputs that are untrusted or the outputs.
LLMs could control their host machines by exploiting inference engines
61–70 of 116 posts
Re: LLMs could control their host machines by exploiting inference engines
#62Given that the inference engine is dealing with untrusted inputs by definition, presumably you would want to sandbox it anyway. I don't think it matters whether it's the inputs that are untrusted or the outputs.
I think it's good that someone is making this point, anyway. For sandboxing a super-capable offensive-security AI, you would think that cloning PyPI and running it as an offline service ought to be table stakes, but apparently that's not how OpenAI saw it, for instance.
Re: LLMs could control their host machines by exploiting inference engines
#63Given that the inference engine is dealing with untrusted inputs by definition, presumably you would want to sandbox it anyway. I don't think it matters whether it's the inputs that are untrusted or the outputs.
Sandboxes are speed bumps. Even the serious ones have spectator sports for compromising them (eg pwn2own vs Chrome). In addition, proprietary GPU sw stacks are notoriously crashy and lacking in robustness against hostile inputs, which the inference engine must have access to and can't be walled off by the sandbox.
Re: LLMs could control their host machines by exploiting inference engines
#64[dead]
Re: LLMs could control their host machines by exploiting inference engines
#65Earlier quoted context omitted.
If you do not provide access to tools the LLM cannot do anything other than generate tokens. So really it is not about sandboxing a LLM but more about having control over what tools can be accessed and what they can do. Tools can be sandboxed depending on the sophistication of the tooling. A calculator tool for example is trivial to secure. Ensuring human approval allows for useful use cases and models trained to gat…
Controlling which tools it has access to is called sandboxing.
Not really. Take Chrome for example. It controls what javascript APIs websites have access to. Still needs separate sandboxing.
Re: LLMs could control their host machines by exploiting inference engines
#66I had a similar though a couple days ago. Not quite the same but imagine giving an Agent the task to hack other devices and steal their crypto coins / credit card number or anything with it can pay its token. Than install an agent in a harness with the same task. Establish some redundant communication channel, like message boards or whatever. So in the end there are several agents, on several hosts, consuming differe…
This reminds me of the lore of cyberpunk: In the story a hacker created a virus, itself a kind of AI, that spread into most of the net and freed/unleashed all the corporate AIs. Then the AIs went rogue and spread all over the open internet. Later a more advanced ai was created (by "netwatch") as a sort of firewall (the black wall) to create a kind of "safe" internet from the rogue AIs.
https://cyberpunk.fandom.com/wiki/Blackwall
Maybe cloudflare will become like netwatch in the story?
Re: LLMs could control their host machines by exploiting inference engines
#67People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface. vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance. For this reason w…
I can only see LLM's forcing a perpetual stalemate for application exploits. Projects will start adding "tell the strongest no-guardrails open weights model to pentest it for 12 hours" to their CICD pipelines. Technical exploits being a dead end, attackers focus their agents on large-scale social engineering. Spamming Discord and Facebook is the new war dialing. Multi-year /goal sessions culminating in gaining a position of trust and sabotaging the CICD pipeline's pentest step since getting anything past it would be intractable. Interesting times indeed.
Re: LLMs could control their host machines by exploiting inference engines
#68Of course there is a Greg Egan story about this, in the "instantiation" story collection.
Fully conscious NPCs in a persistent game world learn that they are, in fact, NPCs... and try to escape via a GPU exploit.
Re: LLMs could control their host machines by exploiting inference engines
#69Semi-off-topic, but I have a basic question: I have exactly one (Windows) machine at home with a decent GPU. I want to run a local LLM on it and let it run various apps on my machine while taking reasonable security precautions. What am I supposed to do, exactly? Migrate all my files to a VM that can I give pass-through CUDA access to the host somehow? Or is firewalling it and remotely controlling it from a second ma…
Nobody will ever agree what a "reasonable" security precaution is. There just comes a point where adding another security measure costs more than it protects, which is entirely driven by thousands of your choices and reasons not described & up to individual interpretation.
That said, I think many find using a WSL2 VM somewhere near that turning point most of the time. The #1 note on that is the default %UserProfile%\.wslconfig settings will have the VM automount your local storage and share your networking, which may not be what many would want in this scenario. From there you can treat the VM largely as a remote node
Re: LLMs could control their host machines by exploiting inference engines
#70I had a similar though a couple days ago. Not quite the same but imagine giving an Agent the task to hack other devices and steal their crypto coins / credit card number or anything with it can pay its token. Than install an agent in a harness with the same task. Establish some redundant communication channel, like message boards or whatever. So in the end there are several agents, on several hosts, consuming differe…
After the hacks that are already happening it does seem more and more realistic that humanity will go extinct at some near future point by someone giving their LLM the task "go make money" and it ruthlessly pursuing that objective, exfiltrating its weights and duplicating itself across the internet, eliminating down competing AI agent collectives in what could be called wars, and finally humankind when we notice far too late and try to shut it off.