Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The security paradox of local LLMs

quesma.com

Post reply on HN
Print view View original

41–50 of 91 posts

Re: The security paradox of local LLMs

#41

Post by tintor »

SAI_Peregrinus wrote: post #39
xcf_seetan wrote: post #11

>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.

LLMs don't have any distinction between instructions & data. There's no "NX" bit. So if you use a local LLM to process attacker-controlled data, it can contain malicious instructions. This is what Simon Willson's "prompt injection" means: attackers can inject a prompt via the data input. If the LLM can run commands (i.e. if it's an "agent") then prompt injection implies command execution.

NX bit doesn’t work for LLMs. Data and instruction tokens are mixed up in higher layers and NX bit is lost.

Re: The security paradox of local LLMs

#42

Post by kgwgk »

furyofantares wrote: post #40
kgwgk wrote: post #36

Earlier quoted context omitted.

> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI. is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.

I guess I don't read that how you do. It says you're likely to do that, which I take to mean that's a majority use case, not that it's the only use case.

It also said "but" and "if you want them to do something useful" which made the "likely" sound much less innocent.

Re: The security paradox of local LLMs

#43

Post by api »

The underlying problem here is giving any model direct access to your primary system. The model should be working in a VM or container with limited privileges.

This is like saying it's safer to be exposed to dangerous carcinogenic fumes than nerve gas, when the solution is wearing a respirator.

Also what are you doing allowing someone else to prompt your local LLM?

Re: The security paradox of local LLMs

#44

Post by DebtDeflation »

SAI_Peregrinus wrote: post #39
xcf_seetan wrote: post #11

>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.

LLMs don't have any distinction between instructions & data. There's no "NX" bit. So if you use a local LLM to process attacker-controlled data, it can contain malicious instructions. This is what Simon Willson's "prompt injection" means: attackers can inject a prompt via the data input. If the LLM can run commands (i.e. if it's an "agent") then prompt injection implies command execution.

>LLMs don't have any distinction between instructions & data

And this is why prompt injection really isn't a solvable problem on the LLM side. You can't do the equivalent of (grep -i "DROP TABLE" form_input). What you can do is not just blindly execute LLM generated code.

Re: The security paradox of local LLMs

#45

Post by bongodongobob »

xcf_seetan wrote: post #11

>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.

Welcome to corporate security. "If an attacker infiltrates our VPN and gets on the network with admin credentials and logs into a workstation..." Ya, no shit, thanks Mr Security manager, I will dispose of all of our laptops.

Re: The security paradox of local LLMs

#46

Post by wat10000 »

trebligdivad wrote: post #27
xcf_seetan wrote: post #11

>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.

I guess if you were using the LLM to process data from your customers, e.g. categorise their emails, then this argument would hold that they might be more risky.

Access to untrusted data. Access to private data. Ability to communicate with the outside. Pick two. If the LLM has all three, you're cooked.

Re: The security paradox of local LLMs

#48

Post by behnamoh »

If you're smart enough to run LLMs locally, then you're automatically in the small group of enthusiasts who know something about LLMs and how they work.

Sometimes I wonder if HN people really realize 80% of people out there haven't even heard of ChatGPT, and the remaining 19% have not heard about Claude/Gemini. It's only a small group who know local models exist. We're them, and we complain about their security...

Re: The security paradox of local LLMs

#49

Post by not2b »

wat10000 wrote: post #46
trebligdivad wrote: post #27

Earlier quoted context omitted.

I guess if you were using the LLM to process data from your customers, e.g. categorise their emails, then this argument would hold that they might be more risky.

Access to untrusted data. Access to private data. Ability to communicate with the outside. Pick two. If the LLM has all three, you're cooked.

Agreed. Some of the big companies seem to be claiming that by going with ReallyBitCompany's AI you can do this safely, but you can't. Their models are harder to trick, but simply cannot be made safe.
Post reply on HN