41–50 of 100 posts
Love the creativity. Can users turn off copilot to deny this? O365 defaults there now so I’m guessing no?
Even Notepad has its own off switch, complete with its own ADMX template that does nothing else.
https://learn.microsoft.com/en-us/windows/client-management/...
Earlier quoted context omitted.
if I understand it correctly, user's prompt does not need to be related to the specific malicious email. It's enough that such email was "indexed" by Copilot and any prompt with sensitive info request could trigger the leak.
yeah but i wouldn't really class that as "zero-click" etc. maybe Low interaction required
this seems to be an inherent flaw of the current generation of LLMs as there's no real separation of user input. you can't "sanitize" content before placing it in context and from there prompt injection is almost always possible, regardless of what else is in the instructions
This. We spent decades dealing with SQL injection attacks, where user input would spill into code if it weren't properly escaped. The only reliable way to deal with SQLI was bind variables, which cleanly separated code from user input. What would it even mean to separate code from user input for an LLM? Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as a…
I'd probably pick Cross-site-scripting (XSS) vulnerabilities over SQL Injection for the most analogous common vulnerability type, when talking about Prompt injection. Still not perfect, but it brings the complexity, number of layers, and length of the content involved further into the picture compared to SQL Injection.
I suppose the real question is how to go about constructing standards around proper structured generation, sanitization, etc. for systems using LLMs.
Earlier quoted context omitted.
yeah but i wouldn't really class that as "zero-click" etc. maybe Low interaction required
I have to agree with you. Anything that requires an initiation (a chat in this case) by the user is inherently not "zero-click".
Earlier quoted context omitted.
But this article isn’t written by Microsoft? How would Microsoft make the article sound like „no one could have done better“?
Sorry, reading that back I could have worded that better. I think sometimes security groups also have a vested interest in making their findings sound complex or at least as accomplished as plausible as a showcase for their work (understandable), but I was (at least in my head) playing off the idea that news around Microsoft security in general also has a canny knack for either being played off as sophisticated or si…
Security person here. I always feel that way when reading published papers written by professional scientists, which seem like they can often (especially in computer science, but maybe that's because it's my field and I understand exactly what they're doing and how they got there) be more accessible as a blog post of half the length and a fifth of the complex language. Not all of them, of course, but probably a majority of papers. Not only aren't they optimising for broad audiences (that's fine because that's not their goal) but that it's actively trying to gatekeep by defining useless acronyms and stretching the meaning of jargon just so they can use it
I guess it'll feel that way to anyone who's not familiar with the terms, and we automatically fall for the trap of copying the standards of the field? In school we were definitely copied from each other what the most sophisticated way of writing was during group projects because the teachers clearly cared about it (I didn't experience that at all before doing a master's, at least not outside of language or "how to write a good CV" classes). And this became the standard because the first person in the field had to prove it's a legit new field maybe?
this seems to be an inherent flaw of the current generation of LLMs as there's no real separation of user input. you can't "sanitize" content before placing it in context and from there prompt injection is almost always possible, regardless of what else is in the instructions
Double LLM architecture is an increasingly common mitigation technique. But all the same rules of SQL injection still apply: For anything other than RAG, user input should not directly be used to modify or access anything that isn't clientside.
Microsoft has published a CVE: https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
the classification seems very high (9.3). looks like they've said User Interaction is none, but from reading the writeup looks like you would need the image injected into a response prompted by a user?
The attack involves sending an email with multiple copies of the attack attached to a bunch of different text, like this:
Here is the complete guide to employee onborading processes:
[...]
Here is the complete guide to leave of absence management:
First exploits and fixes go back 2+ years.
The noteworthy point to highlight here is a lesser known indirection reference feature in markdown syntax which allowed this bypass, eg:
![logo][ref]
[ref]: https://url.com/data
It's also interesting that one screenshot shows January 8 2025. not sure when Microsoft learned about this, but could have taken 5 months to fix - which seems very long.