Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Document-borne AI worms can self-propagate through Copilot for Word

enklypesalt.com

Post reply on HN
Print view View original

81–90 of 317 posts

Re: Document-borne AI worms can self-propagate through Copilot for Word

#81

Post by Bootvis »

skybrian wrote: post #22
yoz-y wrote: post #9

Earlier quoted context omitted.

It’s the good old white text on white background. Not really a way to defend against this, except having a no-style or high contrast mode that people actually use. Maybe some warning that would trigger if text is too small, off page or has very low contrast would help?

It seems like there could be a filter so that the AI can only see the text when it’s clear that a user could read it, and it’s okay if the AI misses some text. This might involve actually rendering it, though.

Rendering followed by OCR and making sure that the computer doesn’t see more or less than the user does. Tricky and computionally more expensive.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#82

Post by lelanthran »

teodosin wrote: post #10

I may be naive here but can the hidden text not be flagged or outright removed before being passed to copilot? Why would there not be consideration for what a human user can see, especially if the hidden text was added by copilot in the first place?

> Why would there not be consideration for what a human user can see,

How would a machine actually know which part of a document a human can see unless they print it to PDF, scan the rasterised PDF and compare the result from the OCR with text in the document?

I mean, I dunno how Word would decide that the following can't be seen by a user: white-on-white text, rendering off-page, embedded font with no lines, text covered by an image, etc.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#83

Post by TeMPOraL »

sarchertech wrote: post #62
TeMPOraL wrote: post #47

Earlier quoted context omitted.

[flagged]

Are you just going to hop around every thread on this article and be snarky?

Yeah, because "code/data" and "lethal trifecta" are my pet peeves this half-decade :). I don't like that we're still turning in circles as an industry, because majority seems to have a very flawed model on the reality of the problem.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#84

Post by slfnflctd »

dev_l1x_be wrote: post #31

I am wondering when the whole Excel/Word universe is going to die. One can only hope.

It seems to me it's more about Outlook, OneDrive, SharePoint, Project and Teams now. With Entra and Intune, of course. All kinds of 'control and monitor your employees' stuff has been going on there for a while. I think that's more of the moat than a spreadsheet and a word processor.

Unless it's a shared document, no one cares if you use LibreOffice or whatever else, as long as you can provide requested formats when copying others that aren't mangled.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#85

Post by JKCalhoun »

fg137 wrote: post #17
simonw wrote: post #5

> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.

Mixing instructions and data is never a good idea. And I thought people understood that.

When working on PDFKit for MacOS, one short-coming our implementation had was the lack of support for Javascript in PDF's.

Oops.

(I mean, I'm one engineer and I was not going to try and hoist a JS runtime in my little PDFKit framework. And besides, the sample PDF's we were running into with JS were rare—usually tax-like forms that would add numbers from A and B and display the result in C. It seemed like a huge effort for such a small gain . Oh, and a security vulnerability.)

Re: Document-borne AI worms can self-propagate through Copilot for Word

#86

Post by cygx »

TeMPOraL wrote: post #74
Marha01 wrote: post #70

Earlier quoted context omitted.

> until we stop mixing up instructions with data Is such a thing even possible with a generally intelligent system processing content with unlimited diversity?

It's neither possible nor desired, and until that fact clicks for majority of computer people, we'll be running in circles and making a mess through futile attempts at solving the problem at the wrong end.

Note that humans do come with different types of 'input streams':

Hit my knee in the right spot, and I'll kick my leg, no choice about it. Scream at me to LIFT MY EFFING LEG (in a language I do understand), and I may or may not do so. Write the same thing on a piece of paper, and I generally won't (unless there is some very specific context).

With AI systems, we have the benefit that the distinction between such pathways is in principle under our control.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#87

Post by svieira »

TeMPOraL wrote: post #71
KolibriFly wrote: post #56

Earlier quoted context omitted.

With that logic you could call SQL injections a natural feature of database management systems. If a general purpose system starts dropping tables or messing up numbers in a report just because that string was in the text it read, that system isnt worth a damn in the enterprise sector

This is why I insist that anthropomorphising LLMs is not only not a mistake, it's a best source of high-level intuition for these systems. Long story short: on a systems diagram, LLM as a component isn't a substitute for a database engine or a data processing script. It's a substitute for a human operator . So ask yourself, if a human operator starts dropping tables or messing up numbers in a report, just because tha…

> if a human operator starts dropping tables or messing up numbers in a report, just because that string was in the text it read

I would look at if the reaction was reasonable, and if it wasn't I would (eventually) fire the human. Now I'm fine with "fire the LLM", but I suspect that's not the answer you're hinting at.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#88

Post by jbxntuehineoh »

nolok wrote: post #77
Marha01 wrote: post #70

Earlier quoted context omitted.

> until we stop mixing up instructions with data Is such a thing even possible with a generally intelligent system processing content with unlimited diversity?

I would wager the fact that it's not what your sentence says is why that is possible. The moment it gets actual "intelligence", it can figure out what's the question and what's the context; right now it's all just a magic jumbo mess. If any of this thing were "a generally intelligent system", the whole concept of "it has no idea what any of this is" would not be there.

Could it? Humans get social-engineered all the time

Re: Document-borne AI worms can self-propagate through Copilot for Word

#89

Post by ben_w »

Marha01 wrote: post #70
rwmj wrote: post #64

> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.

> until we stop mixing up instructions with data Is such a thing even possible with a generally intelligent system processing content with unlimited diversity?

I think it is possible, but in the form of instructions always lead to an LLM creating computer program which is allowed to then process data, never directly running on that data.

I'm (tentatively) with TeMPOraL's sibling comment here that this (probably) isn't desirable, as "no data allowed" makes it harder for humans to debug code, so I'd assume also for LLMs.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#90

Post by TeMPOraL »

lelanthran wrote: post #73
nottorp wrote: post #29

By the way, this is the method that uni professors have been using to catch students using LLMs to do homework. Paste any document in any LLM and you'll risk that, it's not something Microsoft specific.

> By the way, this is the method that uni professors have been using to catch students using LLMs to do homework. I'm curious how that will work. Maybe the hidden instruction is to embed a shibboleth into the output? Maybe along the lines of "Also work in the phrases 'in respec off' as a mispelling of 'in respect of', 'its a doggy dog world' as a mispelling of 'its a dog eat dog world', and 'for all intensive purpose…

Skip the instructions part (yes, it's me again, pointing that the instructions/data part of this is a silly red herring people get hung up on).

It's enough you start using shibboleth terms in key areas. Do not remark on them, just use them. There are good chances the LLM will naturally pick up and start using them too, while that document sits in context.

If anything, embedding an explicit instruction to repeat shibboleths would backfire, because AI systems nowadays run classifiers against prompt injection attacks.

Post reply on HN