Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Document-borne AI worms can self-propagate through Copilot for Word

enklypesalt.com

Post reply on HN
Print view View original

171–180 of 317 posts

Re: Document-borne AI worms can self-propagate through Copilot for Word

#171

Post by skybrian »

layer8 wrote: post #23
skybrian wrote: post #7

Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?

As the sibling comments illustrate, “hidden text” isn’t well-defined, and it has legitimate purposes that end users consciously make use of. The AI needs access to it, for one because the user might actually want the AI to perform actions on the hidden text (not in the sense of following instructions stated in the hidden text, but in the sense of manipulating the hidden text as part of the document), and also because…

Maybe the AI doesn’t need access to it by default? It could be hidden behind another tool call.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#173

Post by ben_w »

watwut wrote: post #149
TeMPOraL wrote: post #126

Earlier quoted context omitted.

You're saying that people fall for phishing because scammers invent completely unrealistic scenarios that would never happen outside TV shows?

I am saying it is unbelievable scenario and yes, I want the person dealing with it ignore it as such.

Then you're making the opposite mistake (but still a mistake) as all officers going in with lethal force during a swatting.

• https://www.nbcnews.com/id/wbna12208992

• https://newsinfo.inquirer.net/1070007/suicidal-caller-mistak...

• https://hongkongfp.com/2026/04/15/woman-trapped-in-tai-po-bl...

• https://en.wikipedia.org/wiki/Triangle_Shirtwaist_Factory_fi...

Re: Document-borne AI worms can self-propagate through Copilot for Word

#174

Post by paganartifact »

cryptonym wrote: post #116
teodosin wrote: post #10

I may be naive here but can the hidden text not be flagged or outright removed before being passed to copilot? Why would there not be consideration for what a human user can see, especially if the hidden text was added by copilot in the first place?

Hide your prompt injection in terms & conditions, plain sight but totally invisible.

[dead]

Re: Document-borne AI worms can self-propagate through Copilot for Word

#175

Post by yoz-y »

TeMPOraL wrote: post #48
yoz-y wrote: post #46

Earlier quoted context omitted.

Only in systems that need to be themselves super generalist. Which is almost never the case.

LLMs are.

LLMs by themselves are but most applications built on top of them are not.

If the code/data separation can not be solved then the whole approach need to be scrapped.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#176

Post by andrewflnr »

iamacyborg wrote: post #65
WJW wrote: post #58

Earlier quoted context omitted.

Security minded programmers understand that. "People" as a whole have not even heard about mixing instructions and data, and certainly not the reasons why it is not a good idea. And AI chatbots are very much targeted at the second group, not the first.

> And AI chatbots are very much targeted at the second group, not the first. I suppose this is why the AI labs are famously not releasing developer-oriented tools.

There's a (terrifyingly) large number of developers who don't qualify as "security-oriented programmers".

Re: Document-borne AI worms can self-propagate through Copilot for Word

#177

Post by bigbuppo »

loumf wrote: post #104
nolok wrote: post #77

Earlier quoted context omitted.

I would wager the fact that it's not what your sentence says is why that is possible. The moment it gets actual "intelligence", it can figure out what's the question and what's the context; right now it's all just a magic jumbo mess. If any of this thing were "a generally intelligent system", the whole concept of "it has no idea what any of this is" would not be there.

Part of reading a document is that in the middle of it, it may ask the reader to do something. That is true for humans too. Sometimes they might not realize that the instructions are malicious or are coerced to comply. A simple example: Let’s say I know that you have a human assistant reading your email, summarizing and filtering it, and then forwarding on the important ones to you. I could write an email that is dir…

Right, but the human assistant could go to prison if they comply with the bribe. Does the CEO of the AI company go to prison if their AI goes on a crime spree?

Re: Document-borne AI worms can self-propagate through Copilot for Word

#178

Post by notnullorvoid »

rwmj wrote: post #64

> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.

Models can be trained with seperate contexts for these things, but the companies with all the resources are so focused on racing to AGI and "scaling laws" that they don't actually care about research into fixing security risks. Fixing those risks would even negatively affect their marketing.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#179

Post by bigbuppo »

Someone wrote: post #100
nolok wrote: post #77

Earlier quoted context omitted.

I would wager the fact that it's not what your sentence says is why that is possible. The moment it gets actual "intelligence", it can figure out what's the question and what's the context; right now it's all just a magic jumbo mess. If any of this thing were "a generally intelligent system", the whole concept of "it has no idea what any of this is" would not be there.

> The moment it gets actual "intelligence", it can figure out what's the question and what's the context; Humans fall for social engineering (“I know you are not allowed to give anybody that information without Id, but I’m your CEO, my phone and passport got stolen,…) I don’t see why AI should be different.

There are two big differences, though. First, humans will generally face consequences for their screwups. Second, AI is doing these screwups at scale while often holding the keys to the kingdom for some idiotic reason.

Re: Document-borne AI worms can self-propagate through Copilot for Word

#180

Post by yuchi »

Schlagbohrer wrote: post #169

Has anyone pointed out yet that in a world full of AIs, these worms are just memes? Memetic idea propagation, same as what happens with us apes.

Those are parasitic memes, that bring no value to the host. There are many such memes in apes world too.
Post reply on HN