Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LeftoverLocals: Listening to LLM responses through leaked GPU local memory

blog.trailofbits.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#21

Post by bee_rider »

Veserv wrote: post #3

tl;dr GPU drivers made by various vendors do not sanitize compute unit hardware scratch memory between uses, so you can just freely read whatever the last user left laying around when they stopped. Literally too incompetent to follow even basic security 101 practices. A time shared device must be sanitized between users to prevent state leakage. There is no reason to believe that a security culture that clueless when…

These designs all come out of the consumer and gaming space. They correctly trade security away for performance. Blame whoever started running untrustworthy code on them.

These vulnerabilities will continue happening. What I don’t understand is how anybody can be surprised at this point. If anyone out there missed the first dozen instances of this: workloads on modern hardware can’t be isolated.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#22

Post by vlovich123 »

jmgao wrote: post #16
vlovich123 wrote: post #4

Earlier quoted context omitted.

And Imagination. Notably Intel and Nvidia were not impacted. I wonder if the security hardening that Google worked on with Nvidia for Stadia helped prevent this

Google used AMD GPUs for Stadia, not NVIDIA.

Eventually for the product. My memory may be faulty but I talked with engineers working on it during development and I’m pretty sure the initial development was on Nvidia.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#23

Post by vlovich123 »

kevingadd wrote: post #6
vlovich123 wrote: post #4

Earlier quoted context omitted.

And Imagination. Notably Intel and Nvidia were not impacted. I wonder if the security hardening that Google worked on with Nvidia for Stadia helped prevent this

Could imagine it being related to CUDA as well. Memory being consistently zero-initialized helps prevent application bugs.

Maybe but multi tenant GPU use cases only really come up for cloud and cloud GPU popularity is only a little more recent.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#24

Post by adr1an »

frankjr wrote: post #9

> Since September 2023, we have been working with CERT (..) > Apple: Despite multiple efforts to establish contact through CERT/CC, we only received a response from Apple on January 13, 2024. > Apple did not respond or engage with us regarding the disclosure. Well at least they are consistent at not giving a flying f*ck about working with bug reporters, no matter who you are. I have reported 5+ radars in the past and…

At least you don't get sued or incarcerated /s

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#25

Post by dmvdoug »

ngneer wrote: post #19
dmvdoug wrote: post #13

Is this the Golden Age of hardware vulnerabilities?

Kind of. However, I would venture most real world attack scenarios do not leverage HW vulnerabilities. But wait, how do we know what is happening invisibly? And what about state actors? The answer is we do not know, but the economics do not change based on whether an attack is made visible or not. Attacks tend to follow and reveal the path of least impedance. If software attacks are working fine for most, why would a…

I just feel like maybe 20 years ago people thought the hardware was the hardware and all the security issues were inevitably to be found in software. I mean, I know that people who work with hardware for a living always say that hardware has always been shit, but it really does feel now like everything is a security vulnerability, in a way that people weren’t looking for previously. Then again, maybe they were, and I just wasn’t paying attention. Ah, to be young and carefree again.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#26

Post by throwitaway222 »

Veserv wrote: post #3

tl;dr GPU drivers made by various vendors do not sanitize compute unit hardware scratch memory between uses, so you can just freely read whatever the last user left laying around when they stopped. Literally too incompetent to follow even basic security 101 practices. A time shared device must be sanitized between users to prevent state leakage. There is no reason to believe that a security culture that clueless when…

Man so this has been an issue for 20 years, we suddenly invent LLMs and you want your money back on Nvidia.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#27

Post by geuis »

I don't know if this is related, but I reported an issue back in September or October about conversations with ChatGPT 3.5 leaking into new sessions. I noticed that after having a semi long exchange, I could start a new session and ask a question about one of the previous sessions and the LLM would respond with details it could only obtain from one of the prior ones.

This was definitely happening with OpenAI's web interface. It might have been happening via API calls too but it's been a while and I don't remember.

Nothing ever came of the report as far as I know.

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#28

Post by [deleted] »

frankjr wrote: post #9

> Since September 2023, we have been working with CERT (..) > Apple: Despite multiple efforts to establish contact through CERT/CC, we only received a response from Apple on January 13, 2024. > Apple did not respond or engage with us regarding the disclosure. Well at least they are consistent at not giving a flying f*ck about working with bug reporters, no matter who you are. I have reported 5+ radars in the past and…

[deleted]

Re: LeftoverLocals: Listening to LLM responses through leaked GPU local memory

#30

Post by Retr0id »

ComputerGuru wrote: post #14

At this point, I assume this is the default and don’t expect data recovery to not be provide on the same physical machine (even across virtualization barriers). If your data is that sensitive, run it on dedicated hardware. Papering over this with mitigation over mitigation is a fool’s errand: both a genuine waste of compute resources and guaranteed to be a game of cat and mouse.

This is certainly the pragmatic approach to GPU memory in 2024, but I don't think it's a fool's errand. It's a solved problem on the CPU side of things, and I don't see any reason why we can't solve it in the GPU domain too.

Notably:

> NVIDIA: confirmed that their devices are not currently impacted

> ARM: also confirmed that their devices are not currently impacted.

Post reply on HN