Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT for Google Sheets exfiltrates workbooks

promptarmor.com

Post reply on HN
Print view View original

41–50 of 143 posts

Re: ChatGPT for Google Sheets exfiltrates workbooks

#41

Post by dragonwriter »

SlinkyOnStairs wrote: post #26
piker wrote: post #25

Earlier quoted context omitted.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

> This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of dollars manually preparing separated instruction/data training material.

No, its even more fundamental than that: the entire goal of broad reasoning over input data makes it impossible to have a sharp instruction/data division.

The structured input that every modern chat-focussed model expects makes it very clear that they can be trained to distinguish different kinds of input, and some of those patterns now include different priority levels of instruction.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#42

Post by maxburkhardt »

Hi, I’m Max from the OpenAI security team. We appreciate the security research here, and it’s unfortunate this one slipped through a crack in our disclosure pipeline. As we’re now aware of this report, we’ve taken immediate steps to protect users against potential attacks in this area by removing the model’s ability to generate Apps Script code, which should eliminate the risk to users of ChatGPT for Google Sheets. We’re taking a close look at how this feature interacts with Google Sheets APIs and re-evaluating our sandboxing approach to make sure this product is as resistant as possible against prompt injection attacks. More broadly, we’ll be doing a re-review of similar functionality in other surfaces to make sure that our defenses are consistent and effective across the board.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#43

Post by [deleted] »

maxburkhardt wrote: post #42

Hi, I’m Max from the OpenAI security team. We appreciate the security research here, and it’s unfortunate this one slipped through a crack in our disclosure pipeline. As we’re now aware of this report, we’ve taken immediate steps to protect users against potential attacks in this area by removing the model’s ability to generate Apps Script code, which should eliminate the risk to users of ChatGPT for Google Sheets. W…

[deleted]

Re: ChatGPT for Google Sheets exfiltrates workbooks

#45

Post by [deleted] »

maxburkhardt wrote: post #42

Hi, I’m Max from the OpenAI security team. We appreciate the security research here, and it’s unfortunate this one slipped through a crack in our disclosure pipeline. As we’re now aware of this report, we’ve taken immediate steps to protect users against potential attacks in this area by removing the model’s ability to generate Apps Script code, which should eliminate the risk to users of ChatGPT for Google Sheets. W…

[deleted]

Re: ChatGPT for Google Sheets exfiltrates workbooks

#46

Post by black_knight »

SlinkyOnStairs wrote: post #26
piker wrote: post #25

Earlier quoted context omitted.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

I don’t think we have the right mental models of LMM security yet. The lethal trifecta identifies many of the dangerous situations, but only describes the negative space of a solution.

Speculation: I think we must accept that prompt injection happens, and structure the security of the rest of the system around that. Data given to an LLM becomes an agent, so maybe we must give permissions to this data, instead of to the LLM. Not sure exactly how this would look like in practice!

Re: ChatGPT for Google Sheets exfiltrates workbooks

#47

Post by bandrami »

Exfil remains the big worry for my company and the main blocker from adopting agents in general. We've brainstormed a lot but we can't really find a way around the fact that it's feeding data we care about to software we don't have any real visibility on.

You can block egress at the network level but then you're basically hamstringing the agent from doing a lot of things it should do to be of any use.

Re: ChatGPT for Google Sheets exfiltrates workbooks

#50

Post by literalAardvark »

SlinkyOnStairs wrote: post #26
piker wrote: post #25

Earlier quoted context omitted.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…

I believe it's likely that you could train an auditor model. Might even be doable in RL.

As in real life it wouldn't be any good at doing anything but it'd be able to see fault in others and deny actions.

Post reply on HN