Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The security paradox of local LLMs

quesma.com

Post reply on HN
Print view View original

31–40 of 91 posts

Re: The security paradox of local LLMs

#31

Post by simonw »

xcf_seetan wrote: post #17
simonw wrote: post #13

Earlier quoted context omitted.

Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI.

Fair enough. Forgive my probably ignorance, but if Claude Code can be attacked like this, doesn’t that means that also foundation LLMs are vulnerable to this, and is not a local LLM thing?

It's not an LLM thing at all. Prompt injection has always been an attack against software that uses LLMs. LLMs on their own can't be attacked meaningfully (well, you can jailbreak them and trick them into telling you the recipe for meth but that's another issue entirely). A system that wraps an LLM with the ability for it to request tool calls like "run this in bash" is where this stuff gets dangerous.

Re: The security paradox of local LLMs

#32

Post by simonw »

knowaveragejoe wrote: post #29
simonw wrote: post #12

If you can get malicious instructions into the context of even the most powerful reasoning LLMs in the world you'll still be able to trick them into outputting vulnerable code like this if you try hard enough. I don't think the fact that small models are easier to trick is particularly interesting from a security perspective, because you need to assume that ANY model can be prompt injected by a suitably motivated att…

Is there any chance your talk was recorded?

It wasn't, but the written version of it it is actually better than what I said in the room (since I got to think a little bit harder and add relevant links).

Re: The security paradox of local LLMs

#34

Post by yalogin »

This is not new right, LLMs are dumb, they just do everything they are told, and so the orchestration before and after the LLM execution holds key. Even without security, ChatGPT or gemini's value is not just in the LLM but the productization of it which is the layers before and after the execution. Similarly if one is executing local LLMs it's imperative to also have proper security rules around the execution.

Re: The security paradox of local LLMs

#35

Post by mbesto »

> Attacker plants malicious prompt in likely-to-be-consumed content.

Is the author implying that some random joe hacker writes a blog with the content. Then a picks up this content thinking its real/valid. A developer within a firm then asks to write something using said LLM references the information from that blog and now there is a security error?

Possible? Technically sure. Plausible? That's ummm a stretch.

Re: The security paradox of local LLMs

#36

Post by kgwgk »

furyofantares wrote: post #20
Der_Einzige wrote: post #15

Earlier quoted context omitted.

No, I'm not going to do those things. I find extreme utility in applications that I can do with an LLM in an air-gapped environment. I will fight and die on the hill that "LLMs don't need the internet to be useful"

Is anyone fighting you on that hill? Someone who finds it useful to have a local llm ingest internet content is not contrary to you finding uses that don't.

> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI.

is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.

Re: The security paradox of local LLMs

#37

Post by simonw »

kgwgk wrote: post #36
furyofantares wrote: post #20

Earlier quoted context omitted.

Is anyone fighting you on that hill? Someone who finds it useful to have a local llm ingest internet content is not contrary to you finding uses that don't.

> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI. is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.

Yeah, I retracted my statement that they can't do anything useful without the internet here: https://news.ycombinator.com/item?id=45670828

Re: The security paradox of local LLMs

#38

Post by wmf »

codebastard wrote: post #2

The security paradox of executing unverified code. If you are executing local malicious/unknown code for reasons you need to read this...

This vulnerability comes from allowing the AI to read untrusted data (usually documentation) from the Internet. For LLMs the boundary between "code" and "data" isn't as clear as it used to be since they will follow instructions written in human language.

Re: The security paradox of local LLMs

#39

Post by SAI_Peregrinus »

xcf_seetan wrote: post #11

>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.

LLMs don't have any distinction between instructions & data. There's no "NX" bit. So if you use a local LLM to process attacker-controlled data, it can contain malicious instructions. This is what Simon Willson's "prompt injection" means: attackers can inject a prompt via the data input. If the LLM can run commands (i.e. if it's an "agent") then prompt injection implies command execution.

Re: The security paradox of local LLMs

#40

Post by furyofantares »

kgwgk wrote: post #36
furyofantares wrote: post #20

Earlier quoted context omitted.

Is anyone fighting you on that hill? Someone who finds it useful to have a local llm ingest internet content is not contrary to you finding uses that don't.

> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI. is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.

I guess I don't read that how you do. It says you're likely to do that, which I take to mean that's a majority use case, not that it's the only use case.
Post reply on HN