31–40 of 100 posts
Earlier quoted context omitted.
if I understand it correctly, user's prompt does not need to be related to the specific malicious email. It's enough that such email was "indexed" by Copilot and any prompt with sensitive info request could trigger the leak.
yeah but i wouldn't really class that as "zero-click" etc. maybe Low interaction required
This reads like it was written to make it sound a lot more complicated than the security failings actually are. Microsoft have been doing a poor job of security and privacy - but a great job of making their failings sound like no one could have done better.
this seems to be an inherent flaw of the current generation of LLMs as there's no real separation of user input. you can't "sanitize" content before placing it in context and from there prompt injection is almost always possible, regardless of what else is in the instructions
What we really need is a completely separate "control language" (Harvard Architecture) to query the latent space but how to do that is beyond me.
https://en.wikipedia.org/wiki/Von_Neumann_architecture
https://en.wikipedia.org/wiki/Harvard_architecture
this seems to be an inherent flaw of the current generation of LLMs as there's no real separation of user input. you can't "sanitize" content before placing it in context and from there prompt injection is almost always possible, regardless of what else is in the instructions
What would it even mean to separate code from user input for an LLM? Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as an opaque string? If we can't even reliably mix untrusted input with code in a language with a formal grammar, I'm not optimistic about our ability to do so in a "vibes language." Try writing an llmescape() function.
[flagged]
This reads like it was written to make it sound a lot more complicated than the security failings actually are. Microsoft have been doing a poor job of security and privacy - but a great job of making their failings sound like no one could have done better.
But this article isn’t written by Microsoft? How would Microsoft make the article sound like „no one could have done better“?
this seems to be an inherent flaw of the current generation of LLMs as there's no real separation of user input. you can't "sanitize" content before placing it in context and from there prompt injection is almost always possible, regardless of what else is in the instructions
This. We spent decades dealing with SQL injection attacks, where user input would spill into code if it weren't properly escaped. The only reliable way to deal with SQLI was bind variables, which cleanly separated code from user input. What would it even mean to separate code from user input for an LLM? Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as a…
That was one of my early thoughts for "How could LLM tools ever be made trustworthy for arbitrary data?" The LLM would just come up with a chain of tools to use (so you can inspect what it's doing), and another mechanism would be responsible for actually applying them to the input to yield the output.
Of course, most people really want the LLM to inspect the input data to figure out what to do with it, which opens up the possibility for malicious inputs. Having a second LLM instance solely coming up with the strategy could help, but only as far as the human user bothers to check for malicious programs.
Earlier quoted context omitted.
This. We spent decades dealing with SQL injection attacks, where user input would spill into code if it weren't properly escaped. The only reliable way to deal with SQLI was bind variables, which cleanly separated code from user input. What would it even mean to separate code from user input for an LLM? Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as a…
> Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as an opaque string? That was one of my early thoughts for "How could LLM tools ever be made trustworthy for arbitrary data?" The LLM would just come up with a chain of tools to use (so you can inspect what it's doing), and another mechanism would be responsible for actually applying them to the input to y…