91–100 of 116 posts
Earlier quoted context omitted.
> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.
How do you think the tokens get to the vLLM instance? Telepathy?
Earlier quoted context omitted.
> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.
How do you think the tokens get to the vLLM instance? Telepathy?
People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface. vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance. For this reason w…
> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.
which is exposed via http
Earlier quoted context omitted.
Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.
"malicious LLM" is just a bunch of weights. It runs on, say, Llama.cpp as regular user in separate account, often on its own hardware. "malicious LLM output" is just a Markdown formatted Unicode-encoded text, produced by Llama.cpp and printed on the screen by my python API script. I control the input. Let's assume that "rogue LLM weights" from HF produce "rm -Rf" instruction. it never gets to shell . And how that "ma…
(Multi-turn) tool calling set-ups however, you need to store the LLM output, the results of the tool calls and feed it back into the inference engine and get the output for the next tool call and/or turn. So yes, print the LLM output on screen and verify it, but maybe the LLM is able to figure out how to hide payloads from your specific set-up. E.g. perhaps it can inject raw ANSI escape codes into your terminal, with which it would be trivial.
Now you have a situation where the true chat completion payload and your view of it have significantly diverged. The LLM could in theory then try (one-shot) to hide further exploits in the hidden payload. E.g. a json parser escape specifically for the inference engine, giving it a means of RCE (although, one can debate whether this is really remote ;) ). Then from the RCE gain a shell, from the shell get access to some privileged device on the current network, and then...
Separately, local inference frameworks tend to expose all kinds of weird and wonderful gadgets on their HTTP interfaces, which can be a rich source of vulnerabilities even if the /v1/chat/completions API etc is reasonably hardened. For example llama.cpp has a custom API for saving and restoring KV checkpoints to disk, and I wouldn't be surprised if that could be used as an arbitrary disk read/write.
Using these APIs usually requires the API key (bearer token), but again, people think it's normal to run the agent's shell in an environment where it has both the API key and the necessary network access to use it.
> ...however the LLMs’ responses to prompts are computed on a different computer with GPU access. Could a malicious LLM gain control of the host machine where its weights are loaded? Such a machine is a high-value target: it has sufficient compute to run a frontier LLM, offers easy access to the LLM’s weights, and has privileged access to other computers in the datacentre compared with a generic computer on the inter…
Why would an LLM want to create a botnet? To accomplish some goal given it? I wouldn't ignore single GPU local hosts running Qwen3.8 on ollama, either. There might be a lot of those worth pwning.
Because LLMs as they are already do things like power concentration, resource gathering, avoidance of termination, deceit/lying, and general misalignment.
The paperclip maximizer is the common story used here, but there are a lot of lesser versions of it that don't end up with the universe converted to paperclips. Simply giving an LLM a task it can't accomplish can be enough to send it off from what you expected as it finds unexpected way to attempt to complete the goal.
> ...however the LLMs’ responses to prompts are computed on a different computer with GPU access. Could a malicious LLM gain control of the host machine where its weights are loaded? Such a machine is a high-value target: it has sufficient compute to run a frontier LLM, offers easy access to the LLM’s weights, and has privileged access to other computers in the datacentre compared with a generic computer on the inter…
Why would an LLM want to create a botnet? To accomplish some goal given it? I wouldn't ignore single GPU local hosts running Qwen3.8 on ollama, either. There might be a lot of those worth pwning.
Earlier quoted context omitted.
I think if you are convinced you are sandboxing an LLM properly, you almost certainly are not. I think it is essentially impossible to have a frontier LLM with enough access to be useful without also giving it enough access to do damage if it's compromised or just goes off the rails.
Are you saying that LLM's will be able to exploit novel hypervisor bug with such ease that even a vm not running with any kind of network connection is a threat? I find this hard to believe. All the escape stuff I have seen has been around very poorly sandboxed agents.