It's a good technical writeup...
...but I'm particularly mesmerized by the spinning cube animations at the bottom of the page. That dithering effect is gorgeous.
EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
91–100 of 100 posts
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#92Love the creativity. Can users turn off copilot to deny this? O365 defaults there now so I’m guessing no?
O365 defaults there now? I‘m not sure I understand. The Copilot we are talking about here is M365 Copilot which is around 30$/user/month. If you pay for the license you wouldn’t want to turn it off would you? Besides that the remediation steps are described in the article and MS also did some things in the backend.
The o365 landing page is now copilot.
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#93Earlier quoted context omitted.
The minimum you can do is not allow the AI to perform actions on behalf of the user without informed consent. That still doesn't prevent spam mail from convincing the LLM to suggest an attacker controlled library, GitHub action, password manager, payment processor, etc. No links required. The best you could do is not allow the LLM to ingest untrusted input.
> The best you could do is not allow the LLM to ingest untrusted input. How would that even work in practice, when an LLM is mostly to be used by a user, which will provide by default, untrusted input?
[deleted]
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#94Earlier quoted context omitted.
I have to agree with you. Anything that requires an initiation (a chat in this case) by the user is inherently not "zero-click".
So zero click is only if you do not use a mouse on your computer or if it works without turning the computer on?
No, zero click requires no interaction from the user. For a hypothetical example simply having a phone on a cellular network and being susceptible to base-band attacks. No interaction needed, just existing.
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#95Earlier quoted context omitted.
This. We spent decades dealing with SQL injection attacks, where user input would spill into code if it weren't properly escaped. The only reliable way to deal with SQLI was bind variables, which cleanly separated code from user input. What would it even mean to separate code from user input for an LLM? Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as a…
Using structured generation (i.e., supplying a regex/json schema/etc.) for outputs of models and tools, in addition to doing sanity checking on the values returned in struct models sent/received from tools, you are able to provide a nearly identical level of protection as SQL injection mitigations. Obviously, not in the worst case where such techniques are barely employed at all, but with the most stringent use of su…
what open source libraries would you recommend to implement these checks?
also do guardrails in the system prompts actually work?
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#96Earlier quoted context omitted.
Well, in the one-LLM case, the input would have no effect on the chain: you'd presumably describe the input format to the LLM, maybe with a few hand-picked example lines, and it would come up with a chain that should be untainted. In the two-LLM case, the chain generated by the ephemeral LLM would have to be considered tainted until proven otherwise. Your "LLM-in-the-loop" case would just be invariably asking for tro…
Maybe I don't understand your idea. I thought it was the LLM deciding what chain of tools to apply for each input. I don't see great accuracy/usefulness for a one time chain of tool generation via LLM that would somehow generalize to multiple inputs without the LLM part of that loop in the future.
not sure how much we can apply this here, but how about specific LLM judges that look for manipulation of I/O?
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#97Earlier quoted context omitted.
the classification seems very high (9.3). looks like they've said User Interaction is none, but from reading the writeup looks like you would need the image injected into a response prompted by a user?
My notes here: https://simonwillison.net/2025/Jun/11/echoleak/ The attack involves sending an email with multiple copies of the attack attached to a bunch of different text, like this: Here is the complete guide to employee onborading processes: [...] Here is the complete guide to leave of absence management: The idea is to have such generic, likely questions that there is a high chance that a random user prompt will…
very cool break down! it looks like it is very hard to defend against those. I am building a customer facing agent and I am looking for lean ways to defend against these attacks
what do you recommedn?
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#98It seems like the core innovation in the exploit comes from this observation: - the check for prompt injection happens at the document level (full document is the input) - but in reality, during RAG, they're not retrieving full documents - they're retrieving relevant chunks of the document - therefore, a full document can be constructed where it appears to be safe when the entire document is considered at once, but c…
these mail should come from an internal account though right? Or is it possible to poison the output from the outside?
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#99Earlier quoted context omitted.
Sorry, reading that back I could have worded that better. I think sometimes security groups also have a vested interest in making their findings sound complex or at least as accomplished as plausible as a showcase for their work (understandable), but I was (at least in my head) playing off the idea that news around Microsoft security in general also has a canny knack for either being played off as sophisticated or si…
> security groups also have a vested interest in making their findings sound complex Security person here. I always feel that way when reading published papers written by professional scientists, which seem like they can often (especially in computer science, but maybe that's because it's my field and I understand exactly what they're doing and how they got there) be more accessible as a blog post of half the length…
agree imho this industry should start to communicate in a much more immediate way with social media and reels - already happening
Re: EchoLeak – 0-Click AI Vulnerability Enabling Data Exfiltration from 365 Copilot
#100Earlier quoted context omitted.
My notes here: https://simonwillison.net/2025/Jun/11/echoleak/ The attack involves sending an email with multiple copies of the attack attached to a bunch of different text, like this: Here is the complete guide to employee onborading processes: [...] Here is the complete guide to leave of absence management: The idea is to have such generic, likely questions that there is a high chance that a random user prompt will…
very cool break down! it looks like it is very hard to defend against those. I am building a customer facing agent and I am looking for lean ways to defend against these attacks what do you recommedn?
I don't know of any 100% reliable fixes for this, and I've been looking for them for nearly three years: https://simonwillison.net/tags/prompt-injection/
Most promising approach right now is this one: https://simonwillison.net/2025/Apr/11/camel/
This paper is useful too: https://simonwillison.net/2025/Jun/13/prompt-injection-desig...