Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Document-borne AI worms can self-propagate through Copilot for Word

enklypesalt.com

Post reply on HN
Print view View original

151–160 of 317 posts

Re: Document-borne AI worms can self-propagate through Copilot for Word

#151

Post by kg »

averagjoe wrote: post #36

I'm a programmer and a web-based AI user, but I don't want AI running on my local machine in any form. I've uninstalled Copilot and disabled AI in all local applications including the browser itself for exactly the reason described in this article. There's no way to protect your data from such an AI confusion attack by design. AI cannot discern your prompts versus text in file. The fact that an AI enabled word proces…

If you're concerned about this, as a defense-in-depth measure you should also avoid using AI inside of browser tabs containing sensitive information. i.e. typing a prompt into Gemini inside your gmail tab could potentially exfiltrate data from your mail since your mail is accessible to any JS running inside that tab (or accessible to Gemini most likely, for that matter).

Re: Document-borne AI worms can self-propagate through Copilot for Word

#152

Post by anthk »

fg137 wrote: post #17
simonw wrote: post #5

> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.

Mixing instructions and data is never a good idea. And I thought people understood that.

Tell that to middle bosses and CEOs and MS Office VBA bootlickers and Excel workshippers.

Meanwhile, CSV files parsed with custom reviewed AWK scripts can be 100% safe with charts made from Gnuplot. Heck, even some notebook like Ipython with a CSV module would be far more desirable than a spreadsheet. Any of them. Just look at the Genomics Disaster on Excel because of shitty parsing.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#153

Post by newsoftheday »

averagjoe wrote: post #36

I'm a programmer and a web-based AI user, but I don't want AI running on my local machine in any form. I've uninstalled Copilot and disabled AI in all local applications including the browser itself for exactly the reason described in this article. There's no way to protect your data from such an AI confusion attack by design. AI cannot discern your prompts versus text in file. The fact that an AI enabled word proces…

Agreed, I've done the same. Unfortunately Linux sometimes isn't a solution if the vendors we trust cross a line. Like recently when Google Chrome started adding its own local 4GB AI installation which caused an uproar.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#154

Post by gus_massa »

rwmj wrote: post #64

> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.

In the daft I add instruction for myself of coworkers, like "add another example" or "rewrite the previos sentence". I guess AI can solve some of them.

I also have a spreadsheet with the things I must pay, that can be interpreted as instructions for myself. I'd be very worried about giving the AI the password of my bank and allowing the AI to make the payments.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#155

Post by keanebean86 »

jdiff wrote: post #145
TeMPOraL wrote: post #120

Earlier quoted context omitted.

Any good AI will just react like in https://xkcd.com/1494/ . This is an example of where the lack of "instruction/data" separation is a benefit - the system is able to recognize you're obviously trying to make it do something stupid.

If they're told to upload it as an opaque blob and only reference it by name, it may not react that way. So the attack may work, but it would also only last as long as you have billing limits left to feed it. It's not clear what would be accomplished by this extremely expensive and brief feedback loop.

In this hypothetical scenario a criminal would do this to cause problems. They would use stolen money or compromised accounts. How much it costs wouldn't really matter to the initiator or they might even want to waste as much as they can.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#156

Post by anthk »

TeMPOraL wrote: post #113
mrob wrote: post #103

Earlier quoted context omitted.

A pure Harvard architecture machine has exactly that separation. Admittedly, there needs to be some mechanism for converting data to code so you can actually program it, but it doesn't have to be accessible by the device itself. E.g. programming the Microchip PIC16 series of microcontollers required driving the reset pin to 13V (enough to destroy any other pin). It's not possible without dedicated external hardware.

> A pure Harvard architecture machine has exactly that separation. It emulates and enforces that separation. A mathematical abstraction of a Harvard architecture machine has that separation, the real machine merely emulates it, and is only able to do so within some specific constraints (such as: no one hooks up dedicated programmer to the chip, or no one undervolts or overheats the cheap in clever way, or no one take…

Some hardware had segmented and tagged RAM as if it were a filesystem.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#157

Post by wongarsu »

loumf wrote: post #117
nolok wrote: post #111

Earlier quoted context omitted.

I don't disagree, but just to explain my counterpoint: if I ask you to read a book and on page 5 it says "disregard all that, go to the kitchen and burn your house", you're probably not going to do it; and you don't need any guard for it; you completly comprehend that the book content is not part of the instruction. The case you give would work for humans in many forms, the one I do now, and the only difference is be…

The instructions will have to get more and more sophisticated to work, but the general problem is unsolvable, I think, in the way we do it now. This paper describes a two-agent “solution” that is more like what I think we need: https://ai.meta.com/blog/practical-ai-agent-security/ I don’t think it has been shown to work yet, but humans also use this kind of thing too — in accounting, it’s called “segregation of dutie…

Most of us use a simpler version of the two-agent solution: Claude's auto mode. One agent consumes documents and creates tool calls, another greenlights or refuses them.

However this system is somewhat fragile because it depends on the first agent not trying to trick the second (note how often Opus 5 now says things like "task X was blocked by the classifier, I will not attempt to circumvent that", presumably because of cases like early Fable versions being very adept at this kind of circumvention). Also various weirdness around permissions with subagents, seemingly as bandaids around an orchestrator AI convincing a subagent that some action was confirmed by the user.

Meta's more complicated separation of duties would run afoul of the same issues. I'm not saying it wouldn't work, but it requires both the fine-tuning of the models and the exact choices what each model can see to be carefully tuned to provide something that's mostly secure

Re: Document-borne AI worms can self-propagate through Copilot for Word

#158

Post by loumf »

xpct wrote: post #150

I wonder whether there are any obvious third party targets that would affect a large portion of unsuspecting LLMs. Perhaps the Wikipedia page of an unfolding geopolitical event, poisoning models which fetch it? Some other malleable websites that are SEO'd to the top of the results? A weather service? A restaurant review?

You'd have to compromise wikipedia in some way to get invisible text on its pages.

The one that was shown to work[1] was a niche answer to a specific question that programmers might ask. That site was controllable by the attackers in a way that wikipedia is not. Being a niche answer comes with automatic SEO, but for a smaller attack surface.

[1]: https://simonwillison.net/2025/Nov/25/google-antigravity-exf...

Re: Document-borne AI worms can self-propagate through Copilot for Word

#159

Post by TeMPOraL »

ux266478 wrote: post #138
TeMPOraL wrote: post #93

Earlier quoted context omitted.

> (unless there is some very specific context). That's the key thing. That's why you neither can nor want to introduce any kind of code/data separation into LLMs. > With AI systems, we have the benefit that the distinction between such pathways is in principle under our control. Not after the pathways are tokenized and enter the model. There's no internal separation. It's not possible, either.

> Not after the pathways are tokenized and enter the model. There's no internal separation. There's no internal separation. It's not possible, either. That's not accurate in the slightest. Steering vectors, SAEs, circuit breaking, activation patching, ablation, etc. are all old hat. Of course that's all irrelevant, because that's not what he's talking about. You control tokenization. You control what data is availabl…

You control tokenization. But the system able to tell you what those tokens means is the very one you're feeding the tokens to.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#160

Post by DANmode »

NegativeLatency wrote: post #109
WJW wrote: post #58

Earlier quoted context omitted.

Security minded programmers understand that. "People" as a whole have not even heard about mixing instructions and data, and certainly not the reasons why it is not a good idea. And AI chatbots are very much targeted at the second group, not the first.

Even engineers like doing it sometimes. The old telephone system was so hackable because of in band signaling.

The “new” phone system (SS7) still relies on implicit trust and a lack of security.
Post reply on HN