Live data from Hacker News

Document-borne AI worms can self-propagate through Copilot for Word

enklypesalt.com

21–30 of 317 posts

Re: Document-borne AI worms can self-propagate through Copilot for Word

#21
post #7

Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?

Because, in the 1990s, you would print out your document before giving it to someone else to read. In those times, sometimes you'd want to include text in the document that shows while you're editing it (e.g. notes to yourself or draft text you might want to refer to later) but not when printed.

I believe you would see hidden text by default (but this was a long time ago and I may have misremembered) when in "normal mode" (later "draft mode" and now removed entirely), which was the default view and showed a long continuous stream of text without the computation expense of calculating page break locations. But when you switch to "print layout mode" (now the usual view unless you're in reading mode) it would be hidden, so you could see what the document would be like printed, unless you explicitly turned on the display of hidden text in that mode.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#22
post #9
post #7

Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?

It’s the good old white text on white background. Not really a way to defend against this, except having a no-style or high contrast mode that people actually use. Maybe some warning that would trigger if text is too small, off page or has very low contrast would help?

It seems like there could be a filter so that the AI can only see the text when it’s clear that a user could read it, and it’s okay if the AI misses some text. This might involve actually rendering it, though.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#23
post #7

Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?

As the sibling comments illustrate, “hidden text” isn’t well-defined, and it has legitimate purposes that end users consciously make use of. The AI needs access to it, for one because the user might actually want the AI to perform actions on the hidden text (not in the sense of following instructions stated in the hidden text, but in the sense of manipulating the hidden text as part of the document), and also because otherwise it might cause breakage in the document if the AI doesn’t consider the presence of the hidden text when manipulating the document.

What AI tools really need is reliable power-user levels of awareness about Word features, and corresponding structured access.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#24
post #17
post #5

> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.

Mixing instructions and data is never a good idea. And I thought people understood that.

Isn't React, the most popular JS library, an example of that? Clearly people don't understand that

Re: Document-borne AI worms can self-propagate through Copilot for Word

#27
post #5

> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.

Indeed - but some models are more robust than others. I tried to make Opus-5 execute hidden instructions embedded a picture using steganography. It's very hard to find a reliable payload.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#28
post #17

Earlier quoted context omitted.

Mixing instructions and data is never a good idea. And I thought people understood that.

Isn't React, the most popular JS library, an example of that? Clearly people don't understand that

No it’s not an example of that. Do you store components in your component state?

Re: Document-borne AI worms can self-propagate through Copilot for Word

#30
post #25

3 months from first contact to... nothing. Surely those big corps peddling AI dev can't be taken seriously.

Microsoft, and MSRC in particular, have been hands-on and very responsive from the get-go. I think this problem is better viewed as a current LLM technology problem in general. Several mitigations have already been implemented that dramatically reduce the attack surface and propagation frequency. However, in general I think this is a real problem with no real solution yet.
Post reply on HN