> LLMs could
This is going to end up like the Law of Headlines, isn't it? "Do x, y, z Cure All That Ails You?" ... no but we got you to read the article. "LLMs _could_ x, y, z" ... but they don't because they're programs, not magic.
LLMs could control their host machines by exploiting inference engines
21–30 of 116 posts
Re: LLMs could control their host machines by exploiting inference engines
#22This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
I think if you are convinced you are sandboxing an LLM properly, you almost certainly are not. I think it is essentially impossible to have a frontier LLM with enough access to be useful without also giving it enough access to do damage if it's compromised or just goes off the rails.
If you do not provide access to tools the LLM cannot do anything other than generate tokens. So really it is not about sandboxing a LLM but more about having control over what tools can be accessed and what they can do. Tools can be sandboxed depending on the sophistication of the tooling. A calculator tool for example is trivial to secure. Ensuring human approval allows for useful use cases and models trained to gate permissions work. A super intelligence with a weaker approval gate will be able to subvert. Inversely a super intelligent gate should be expected to prevent subversion by a weaker model.
Re: LLMs could control their host machines by exploiting inference engines
#23you would have to be especially incompetent to give a compromise opportunity to streamed tokens, the CVE he listed proves the point. whoever is responsible for that has no business coding anything.
> offers easy access to the LLM’s weights
Re: LLMs could control their host machines by exploiting inference engines
#24The inference engine itself does not execute anything. The agent loop is what may execute a command. So I think this article is a kind of strange. Or maybe the author means that a prompt could potentially mess up the inference. But I find it hard to see how that could take control over the host.
It's more about LLM hacking the inference engine itself from inside. It's an attack surface like any other -- untrusted input goes it, bugs in the parser/tokenizer/API surface lead to an RCE, then it magically tweaks the alignment weights. Boom, somebody finally nukes **sia. Then will never see it coming.
I don't think it's any more probable than other AGI nonsense basilisks included, but it's technically a possibility.
Re: LLMs could control their host machines by exploiting inference engines
#25This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
Also, operating systems should let us set filesystem permissions per app/process/executable instead of just user accounts. Similar to how macOS/iOS Sandboxing works but at a more lower and granular level
personally I wish the OS would allow syscall filtering per user
Re: LLMs could control their host machines by exploiting inference engines
#26Interesting breakdown of a hypothetical attack. The complexity of modern inference engines and the rush to develop them do create some attack surface. But overall it reads more like a "what if" thought experiment.
Splitting the GPU and parser is technically doable, but in practice it's trickier, large models run on clusters where the boundaries between components get blurry so defending against this kind of thing would probably require some serious rethinking of the whole architecture I think.
Re: LLMs could control their host machines by exploiting inference engines
#27This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
If we are treating ai agents like people, you could also just get the AI a laptop and apply the traditional tools to manage user laptops
Re: LLMs could control their host machines by exploiting inference engines
#28This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
[deleted]
Re: LLMs could control their host machines by exploiting inference engines
#29People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface.
vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance.
For this reason we run vLLM on a separately sandboxed VM on a firewalled VLAN. Software updates and models (from Dev/Test env) get pushed onto Prod from an external cache, machine syslog, nvidia load monitoring and vLLM query telemetry out to their loggers, but that is all. No DNS, no AD/LDAP, nothing. Firewall on hosts and VM hosts. Log and telemetry processing done on a completely separate set of VMs in their own isolated subnet, producing reports and alerts that are tightly formatted.
Re: LLMs could control their host machines by exploiting inference engines
#30they could yea...