Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT for Google Sheets exfiltrates workbooks

promptarmor.com

Post reply on HN
Print view View original

31–40 of 143 posts

Re: ChatGPT for Google Sheets exfiltrates workbooks

#31

Post by osigurdson »

dvt wrote: post #11

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…

Does containerization help much here? If it's a code tool then presumably it needs access to your code files (read / write). Maybe there are use cases for it of course.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#32

Post by zmmmmm »

dvt wrote: post #11

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour

I share your concern but it's not a correct characterisation to say they are not taking it seriously:

https://www.anthropic.com/engineering/how-we-contain-claude

My concern is people aren't even addressing this at the right level. People are currently thinking at the level of "how do I build a VM to contain this one agent" when this is actually a "design a whole new OS" level problem.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#33

Post by e12e »

How long did it take from the first macro virus until the industry accepted that "we can't have nice things (at this cost to security)" - macros were defaulted to off everywhere?

How long until the industry accept the risk LLMs pose with "prompt injection"?

Re: ChatGPT for Google Sheets exfiltrates workbooks

#34

Post by zmmmmm »

HPsquared wrote: post #30
dvt wrote: post #11

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…

Local and containerised, without internet access.

effectively, that means it's a VM not a container

because sharing the kernel ultimately means all the devices come along for the ride which give all kinds of fancy ways to communicate with the outside world - network is just the start

I think micro-VMs are the future here, but they need heavy adaptation from their current usage.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#36

Post by bnjemian »

SlinkyOnStairs wrote: post #26
piker wrote: post #25

Earlier quoted context omitted.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

It’s a huge problem, but I’d caution against this absolutism — there may well be structure that can be created around and between LLMs and their outputs to enable the necessary segregation.

As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure).

LLMs aren’t there yet, but it’s entirely plausible that structures may can be developed to solve the problem, and those structures aren’t known or commonly conceived of in the present.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#37

Post by dvt »

osigurdson wrote: post #31
dvt wrote: post #11

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…

Does containerization help much here? If it's a code tool then presumably it needs access to your code files (read / write). Maybe there are use cases for it of course.

WASI provides a very nice mental model where you can mount, e.g., /input, as read-only, and where every mutation is saved in /output or what-not. At least that's my favorite contract: input files remain untouched, but we can copy them and do whatever we want with them in /scratch or /output (which the user can later investigate and make sure nothing went horribly wrong while still having backups).

Re: ChatGPT for Google Sheets exfiltrates workbooks

#38

Post by ethin »

SlinkyOnStairs wrote: post #26
piker wrote: post #25

Earlier quoted context omitted.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

If only there was a language which allowed one to express instructions for a computer to execute which was nearly unambiguous, precise, deterministic, and containerized such that the computer would do exactly what you told it to.

...

Oh wait.

Yes, the above was referring to programming languages. Which is what prompts are, essentially. It's just a different (and more verbose) way of instructing the computer on what to do. It also has a solution space of infinity and is ambiguous enough that there is no way to secure it because there are infinite combinations of saying anything imaginable. All prompt injections do is prove this point, over and over and over again, and "prompting" an LLM is just reverse-engineering programming languages in the worst possible way. I suspect that we will eventually have no other choice but to revert to using programming languages because they are the only way to get the kind of protections that people are trying to come up with with all these containerization and virtualization systems (which inevitably fail).

Re: ChatGPT for Google Sheets exfiltrates workbooks

#39

Post by int3trap »

dvt wrote: post #11

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…

Got a link to your project? I'm working on something that could make use of something like this.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#40

Post by dmoy »

bnjemian wrote: post #36
SlinkyOnStairs wrote: post #26

Earlier quoted context omitted.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

It’s a huge problem, but I’d caution against this absolutism — there may well be structure that can be created around and between LLMs and their outputs to enable the necessary segregation. As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure). LLM…

> As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure)

The better comparison on bit errors would be e.g. rowhammer, an adversarial bit error. Which you absolutely can't ignore.

Post reply on HN