61–70 of 317 posts
Look on the positive side, the faster AI causes more harm the faster our bosses might wake up and push anti-AI company policies! Oh who am I kidding, ya'll asked for this reality. I will take great joy in the suffering from my AI-less soapbox.
Earlier quoted context omitted.
Separation of instructions and data is artificial. Reality has no such separation. A general purpose system needs not to have them either; it's a design feature, not a bug. People get too hung up on this fundamentally wrong idea, and the space of security, instead of progressing, is just running in circles like a headless chicken, making a mess of everything.
Literally all of software is artificial? Being explicit and reasoned about how you choose to allow or deny a particular computation is, surely, at the heart of a lot of computer security?
There's an endless supply of people who think they know what is Code and what is Data, and they're always arguing with others who also think that, and neither realize that Code/Data classification is an opinion, a perspective. It doesn't hold in general.
Having a separation like this makes sense for super narrow systems, where you can define the allowed and disallowed use cases, enforce the distinction (because it's not real - therefore you have to enforce it mechanistically within your system), and willing to accept that some useful operations will be denied by your system.
Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.
Earlier quoted context omitted.
Mixing instructions and data is never a good idea. And I thought people understood that.
Security minded programmers understand that. "People" as a whole have not even heard about mixing instructions and data, and certainly not the reasons why it is not a good idea. And AI chatbots are very much targeted at the second group, not the first.
I suppose this is why the AI labs are famously not releasing developer-oriented tools.
> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.
Mixing instructions and data is never a good idea. And I thought people understood that.
In the GPT-2 era LLMs were just data. Instructions did not exist, and if you added them to your data they would not be followed. Then around 2022 we figured out how to patch in instruction following with a bit of fine tuning, leading to the current AI bubble. That's an ugly hack that leads to all these issues. But it's what this entire AI bubble is founded on. And nobody seems to have found a better way (or at least one that actually scales and doesn't make unreasonable sacrifices)
> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents. Oh no.
Mixing instructions and data is never a good idea. And I thought people understood that.
It's increasingly clear that AI needs to be heavily regulated to be safe for public use. It needs to grow out of it's "wild west" model.
This is equivalent of sql injection and normal worm.
> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.
Is such a thing even possible with a generally intelligent system processing content with unlimited diversity?