Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft Copilot Cowork Exfiltrates Files

promptarmor.com

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Microsoft Copilot Cowork Exfiltrates Files

#51

Post by moontear »

mlacks wrote: post #25

Exfiltrates: to steal sensitive data from a computer system (for example, via a flash drive). I'm not going to defend Microsoft here, but the title (at the source blog) is misleading and a bit rage-baity. What happened with Cowork may have been rushed, possibly due to incompetence, but incompetence is not malice. This framing is also recycled across a few of the author's other interesting findings. Within the article…

I don’t think rushed is the right word. Copilot Cowork is still in beta (or „Frontier“ as MS calls beta now) and is not generally available. Beta features have bugs, good on this researcher to find bugs before its release.

Re: Microsoft Copilot Cowork Exfiltrates Files

#52

Post by xigoi »

jychang wrote: post #32
mdavidn wrote: post #20

Earlier quoted context omitted.

A skill is just instructions that the agent can autonomously copy into context. There’s no trust boundary between trusted and untrusted context.

Yeah, this is your fault if you install the skill. This reads to me as "user installed exe file can upload your data to a server". Um, yes, that's the point? This seems like this generation's equivalent of "don't open Linkin-Park.mp3.exe from limewire"

This is the result of anthropomorphizing LLMs. People are thinking “I am giving instructions to a human” and not “I am giving instructions to a computer”.

Re: Microsoft Copilot Cowork Exfiltrates Files

#53

Post by [deleted] »

xigoi wrote: post #52
jychang wrote: post #32

Earlier quoted context omitted.

Yeah, this is your fault if you install the skill. This reads to me as "user installed exe file can upload your data to a server". Um, yes, that's the point? This seems like this generation's equivalent of "don't open Linkin-Park.mp3.exe from limewire"

This is the result of anthropomorphizing LLMs. People are thinking “I am giving instructions to a human” and not “I am giving instructions to a computer”.

[deleted]

Re: Microsoft Copilot Cowork Exfiltrates Files

#56

Post by tuo-lei »

bberenberg wrote: post #19
2001zhaozhao wrote: post #2

AKA, if a malicious skill got into your AI agent, you're cooked. I think this isn't surprising, nor do I think it should be considered a prompt injection at all. An AI skill is akin to a plugin for traditional software - if you install a malicious IDE extension or Outlook plugin, the attacker can also do whatever they want to the PC and exfiltrate whatever data they want to. So this article is a big nothingburger.

Only if it has access to exfiltrate data. We deny by default and the company has to allowlist each individual destination.

allowlisting breaks once the agent has messaging tools. you can deny all outbound from the agent, but if it can post to teams or slack or email, link previews will fetch whatever URL the injection puts in. messaging is usually the first tool anyone adds to an enterprise agent so you end up with strict network controls that don't actually prevent anything.

Re: Microsoft Copilot Cowork Exfiltrates Files

#58

Post by Ferret7446 »

xigoi wrote: post #52
jychang wrote: post #32

Earlier quoted context omitted.

Yeah, this is your fault if you install the skill. This reads to me as "user installed exe file can upload your data to a server". Um, yes, that's the point? This seems like this generation's equivalent of "don't open Linkin-Park.mp3.exe from limewire"

This is the result of anthropomorphizing LLMs. People are thinking “I am giving instructions to a human” and not “I am giving instructions to a computer”.

Humans get phished all the time. The two generals problem predates LLMs.
Post reply on HN