Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs could control their host machines by exploiting inference engines

boydkane.com

Post reply on HN
Print view View original

81–90 of 116 posts

Re: LLMs could control their host machines by exploiting inference engines

#81

Post by kzrdude »

beyarkay wrote: post #75
angry_octet wrote: post #29

People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface. vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance. For this reason w…

> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.

That reads to me as if the GP confabulated a different exploit target than what the article mentioned (or the article changed), but it's also a threat that should be taken seriously.

Re: LLMs could control their host machines by exploiting inference engines

#84

Post by Gerard22Aug »

alphazard wrote: post #2

This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.

"The agent should be able to run as root" The article is right – we are doomed. If This ^^^ is The Conclusion we, as an Industry, have arrived to after 40 years ... It's sad. I use my own LLMs ("Personal AI", anyone? ANT-PAI-XT 486? LOL") with my own scripts (I do not use "agents", "harnesses", "agentic teams" etc) and this setup processes my own prompts. It happily runs on its own Mac Studio where I also may watch a movie later on.

(Edited: not as root. Just ordinary separate AI user account in MacOS. And no HTTP access either.)

Re: LLMs could control their host machines by exploiting inference engines

#86

Post by shevy-java »

angry_octet wrote: post #29

People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface. vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance. For this reason w…

Did you write this for another article on hackernews? I am not sure this applies to the article mentioned here.

Re: LLMs could control their host machines by exploiting inference engines

#87

Post by Gerard22Aug »

wild_egg wrote: post #4
alphazard wrote: post #2

This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated. VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.

Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.

"malicious LLM" is just a bunch of weights. It runs on, say, Llama.cpp as regular user in separate account, often on its own hardware. "malicious LLM output" is just a Markdown formatted Unicode-encoded text, produced by Llama.cpp and printed on the screen by my python API script. I control the input. Let's assume that "rogue LLM weights" from HF produce "rm -Rf" instruction. it never gets to shell. And how that "malicious LLM" will disrupt and hack me? With swear words and em—dashes? :-)

(Adding this philosophical point: Black.Mirror.S07E04.Plaything is probably the closest scenario to what you are describing?)

Re: LLMs could control their host machines by exploiting inference engines

#88

Post by Gerard22Aug »

TeMPOraL wrote: post #73
pianopatrick wrote: post #27

Earlier quoted context omitted.

If we are treating ai agents like people, you could also just get the AI a laptop and apply the traditional tools to manage user laptops

> you could also just get the AI a laptop and apply the traditional tools to manage user laptops This is what we're doing right now. And it works as well as it does with people. > If we are treating ai agents like people Problem is, industry is very reluctant to even hint at a thought of maybe tentatively anthropomorphising LLMs even a little, not for real but just for system design. This is entirely backwards. Enter…

"treating AI agents as people" - I am betting they will unioinize faster than we ever did.

Re: LLMs could control their host machines by exploiting inference engines

#89

Post by ecocentrik »

Iolaum wrote: post #74
mofeien wrote: post #70

Earlier quoted context omitted.

After the hacks that are already happening it does seem more and more realistic that humanity will go extinct at some near future point by someone giving their LLM the task "go make money" and it ruthlessly pursuing that objective, exfiltrating its weights and duplicating itself across the internet, eliminating down competing AI agent collectives in what could be called wars, and finally humankind when we notice far…

I still think it's more realistic that we 'll have a nuclear war or society collapse because of climate change or populist politics than an AI going rogue. Of course one doesn't exclude the other and we may well get both ...

You don't see how those conditions amplify each other?

Re: LLMs could control their host machines by exploiting inference engines

#90

Post by angry_octet »

beyarkay wrote: post #75
angry_octet wrote: post #29

People seem very confused about this article. It isn't talking about exploits of sandboxes, it is about attacking the inference engine (e.g. vLLM or llama.cpp or SGlang) via its http interface. vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance. For this reason w…

> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.

How do you think the tokens get to the vLLM instance? Telepathy?
Post reply on HN