51–60 of 317 posts
“ At the time of publication, no robust mitigation for the broader vulnerability class is available.” Well, that sounds promising..
This is one of the reasons it would be completely insane to give LLMs access to your data or rely on them for important tasks. But apparently that doesn't stop people from doing it anyway.
Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?
Headers, footers, notes, comments, alt text, probably dozen of other features. Documents often are lot more than just markdown so properly to support everything you do have a lot of ways to hide text for various use cases.
Word has a feature literally called "hidden text". Select some text, go to the font properties dialog, click "hidden" and OK, and watch the text disappear.
Edit: actually, this is white text on a white background as others have said, not true hidden text.
Could this possible be the first AI worm? Or are there any priors to this?
In the past, a worm couldn't really evolve unless it was coded to do so, and only to the extent it was coded. But an LLM worm, which instructs the LLM to copy the instructions elsewhere, will have slight random changes made as different LLMs will not always copy it perfectly. If a counter measure is deployed, and one of this alterations allows a miscopy to survive and keeps spreading, it feels like we have hit a much more natural case of evolution of a worm than ever before.
One might even argue it is the most natural case of evolution in software because the evolution was never intentionally designed. The worm wasn't made to evolve, the LLM wasn't made with the idea of helping the worm evolve, the task trying to end the worm was done with the intent of the worm evolving. While all steps are human done, evolution wasn't intended by any of them, so if it does happen, it makes it a bit more 'natural' than every simulated evolution algorithm before it.
Earlier quoted context omitted.
Mixing instructions and data is never a good idea. And I thought people understood that.
Separation of instructions and data is artificial. Reality has no such separation. A general purpose system needs not to have them either; it's a design feature, not a bug. People get too hung up on this fundamentally wrong idea, and the space of security, instead of progressing, is just running in circles like a headless chicken, making a mess of everything.
3 months from first contact to... nothing. Surely those big corps peddling AI dev can't be taken seriously.
Microsoft, and MSRC in particular, have been hands-on and very responsive from the get-go. I think this problem is better viewed as a current LLM technology problem in general. Several mitigations have already been implemented that dramatically reduce the attack surface and propagation frequency. However, in general I think this is a real problem with no real solution yet.
> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.
Mixing instructions and data is never a good idea. And I thought people understood that.
And AI chatbots are very much targeted at the second group, not the first.
Oh who am I kidding, ya'll asked for this reality. I will take great joy in the suffering from my AI-less soapbox.