Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs could control their host machines by exploiting inference engines

boydkane.com

Post reply on HN
Print view View original

51–60 of 116 posts

Re: LLMs could control their host machines by exploiting inference engines

#51

Post by ma2kx »

I had a similar though a couple days ago. Not quite the same but imagine giving an Agent the task to hack other devices and steal their crypto coins / credit card number or anything with it can pay its token. Than install an agent in a harness with the same task. Establish some redundant communication channel, like message boards or whatever. So in the end there are several agents, on several hosts, consuming different APIs / LLMs and communicating with each other over different channels. Basically the same concept as OpenAI explained when their LLM hacked huggingface but in this scenario their not bound to a single sandboxed environment but spread over the internet. If such a swarm has reached a critical mass it would be pretty dificult to erase them as its impossible to control every inference engine or LLM API endpoint.

In the end its the next evolution step from computer viruses, worms and trojans. So I propose we will call those "ghosts". I.e. a ghost is when a rogue llm takes control over a victims host.

Re: LLMs could control their host machines by exploiting inference engines

#52

Post by chaps »

genxy wrote: post #42

I thought they were going to get the LLM to "think really hard about rowhammer" and have the LLM conjure a JIT.

Same. Was thinking the other week what would be the smallest llm one could make that is able to figure out tooling in its local environment and build something that can then expand out to other hosts, build more of itself, etc.

1MB?

Re: LLMs could control their host machines by exploiting inference engines

#54

Post by dare944 »

pianopatrick wrote: post #25
Razengan wrote: post #3

Earlier quoted context omitted.

Also, operating systems should let us set filesystem permissions per app/process/executable instead of just user accounts. Similar to how macOS/iOS Sandboxing works but at a more lower and granular level

personally I wish the OS would allow syscall filtering per user

I use seccomp filters on linux in my AI sandbox.

Re: LLMs could control their host machines by exploiting inference engines

#55

Post by genxy »

chaps wrote: post #52
genxy wrote: post #42

I thought they were going to get the LLM to "think really hard about rowhammer" and have the LLM conjure a JIT.

Same. Was thinking the other week what would be the smallest llm one could make that is able to figure out tooling in its local environment and build something that can then expand out to other hosts, build more of itself, etc. 1MB?

So digital eBola. If it only uses Apple devices, it would be iBola.

Probably more in the 500-900MB range.

Re: LLMs could control their host machines by exploiting inference engines

#56

Post by pianopatrick »

dare944 wrote: post #54
pianopatrick wrote: post #25

Earlier quoted context omitted.

personally I wish the OS would allow syscall filtering per user

I use seccomp filters on linux in my AI sandbox.

Yeah, I just wish seccomp worked per user. So you could define a policy of which users can do which syscalls, and then that follows them no matter which application they start.

Re: LLMs could control their host machines by exploiting inference engines

#57

Post by ulam2 »

> vLLM and SGLang are complex, and bugs are common This for me is the heart of the issue. Feature creep will lead to the downfall of all these frameworks. Today, we can conjure our own bespoke inference engine for our own hardware in no time. It need only support a few modern model architectures. The code can be audited too. I think the article highlights an important gap area for the industry.

Re: LLMs could control their host machines by exploiting inference engines

#59

Post by AdieuToLogic »

nokcha wrote: post #44
dataflow wrote: post #31

Semi-off-topic, but I have a basic question: I have exactly one (Windows) machine at home with a decent GPU. I want to run a local LLM on it and let it run various apps on my machine while taking reasonable security precautions. What am I supposed to do, exactly? Migrate all my files to a VM that can I give pass-through CUDA access to the host somehow? Or is firewalling it and remotely controlling it from a second ma…

I'd guess that prompt injection is the biggest risk in this setup, dwarfing the risk of exploits against the inference engine. Personally, I run LLM agents only inside a Docker container that limits the LLM's access to sensitive information and the LLM's ability to take irreversible destructive actions. See also: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

> I'd guess that prompt injection is the biggest risk in this setup ...

LLM poisoning[0] would be a much greater risk in a locally executed LLM than prompt injection, given that the LLM would be in an entirely controlled environment.

0 - https://www.anthropic.com/research/small-samples-poison

Re: LLMs could control their host machines by exploiting inference engines

#60

Post by scorpioxy »

dataflow wrote: post #31

Semi-off-topic, but I have a basic question: I have exactly one (Windows) machine at home with a decent GPU. I want to run a local LLM on it and let it run various apps on my machine while taking reasonable security precautions. What am I supposed to do, exactly? Migrate all my files to a VM that can I give pass-through CUDA access to the host somehow? Or is firewalling it and remotely controlling it from a second ma…

Depends on your risk appetite I'd say. But the most straight forward set up that I believe gives you a decent amount of protection would be to use a VM to host the LLM and then execute any agents that would be running the various apps on your machine via a sandbox with access to only the things it needs.

There are many variations to that(firewalls, sandbox abilities etc) but it's a good start in my opinion. And, most importantly, is a far cry from all the people I read about running agents on their machines with admin access and access to their emails and calendars and lives.

Post reply on HN