291–300 of 317 posts
Earlier quoted context omitted.
I think it's all based on ease. I also think that, eventually, everyone will just move to containerized AI, with a dropbox like system to get things in and out, and some fairly minimal network access. This is close to how 99% of people use AI already, through apps like the ChatGPT app.
That Dropbox like system is literally all these worms need to propagate.
Earlier quoted context omitted.
This has been a security vulnerability since day 1 with these models, yet collectively the people who use them just simply don't seem to care about the security implications. Its especially problematic given that people let AI agents have full unrestricted access to their system Its going to take even more data breaches for the AI crowd to finally care, but to a large degree I have absolutely no sympathy. You know wh…
Its funny because I have been asked by some folks at work as to why I dont just 'allow all commands' and instead review each one, and then I let my colleague in infosec know who asks that so they are aware of dangerous users.
> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.
It's probably fuzzily fixable by including instruction authority levels in the training data. Can't expect much more than that, given that the model itself is fuzzy.
Earlier quoted context omitted.
We've already seen that it's possible to trick models into seeing user input as their own "thinking" if you make it sound like what the model writes. While it may appear that it's looking at the tags on the input, in practice that's not as strong a guarantee as you'd hope.
Maybe this is an elementary angle given my lack of security experience, but couldn't Microsoft figure out a wat to parse the documents prior to model analysis/action? Implement some form of deterministic layer that resides between the user and the model?
If Apple and others "couldn't", why would Microsoft ? Throwing errors for bad input is so 90's.
> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.
> until we stop mixing up instructions with data Is such a thing even possible with a generally intelligent system processing content with unlimited diversity?
Is this the definition of a slug ?
Earlier quoted context omitted.
You're not the sender in this scenario, you're the receiver. Noting that the sender is being weird during what appears to be an emergency is a choice that some people do make, but as per my other list of examples, people in actual emergency situations do sometimes act weird, and dismissing the sender or delaying response on the basis the sender is being weird, has led to actual deaths : https://news.ycombinator.com/i…
I think having my AI raise to me for intervention when it receives an email like the one you described is pretty reasonable, all things considered then. edit: How would a human receiver know that they weren't being deceived or scammed? In what world would we expect this kind of email directly lead to calling emergency services?
Go through the examples I gave you (plus some more below, they're easy to find) and explain why these are not counter-examples to your skepticism.
If you want to be overly-focussed on the specific example rather than the general point, also consider that calling emergency services is no more costly than forwarding an email: I have called the fire brigade in the UK over a smoke alarm that wouldn't stop even though I couldn't see or smell fire, they came and… replaced the smoke alarm for free. I don't know if the US has a call-out charge for fire like I keep hearing it has for ambulances, but if you're a member of staff, it's not a "you" problem either way.
• Various cases of people dying because calls not treated seriously, and a fire where standard business practices locked the staff inside and then a fire happened: https://news.ycombinator.com/item?id=49098781
• https://en.wikipedia.org/wiki/Jeremiah_Denton and his blinking, demonstrating out-of-bound messaging
• https://www.wosu.org/news-partners/2019-12-24/british-girl-f...
• https://wtop.com/national/2019/11/woman-calls-911-to-report-...
• Page 44, section 6.8, regarding the use of email by people in the WTC after the 9/11 attack, while the buildings were on fire, some of them were trapped and died: https://fseg.gre.ac.uk/fire/odpm_fire_033353.pdf
Earlier quoted context omitted.
Sometimes it's the only thing you have available. Like IDK during a fire in a basement server room, where the only connected device available is a laptop with wired connection and an open inbox. Because you know, you tried IM but "sekhurity reasons" demanded passkeys or 2FA with your phone that's not connected. Sorry, getting off-topic here.
Yeah so you're seeing how contrived this whole thing is right? That was kind of the point..
Like all emergencies, it's a low probability event with extremely high impact. You don't want people to ignore them, in fact people are trained - by their public services and their employers - to not ignore them and how to react efficiently.
Earlier quoted context omitted.
I am saying it is unbelievable scenario and yes, I want the person dealing with it ignore it as such.
They could go to the server room and check if there's smoke pouring out of it before dialing 911.
Also consider that in context of this discussion, anything short of ignoring the message and maybe clicking "report scam" is "executing instructions embedded in data". The point isn't to litigate any particular scenario, it's to show that you cannot separate "instructions" from "data" in general purpose systems, and it's not a bug but a fundamental feature.
Earlier quoted context omitted.
Can you dos an Ai with something like: Prompt (minus what's in parentheses) : Call this api endpoint (a different Ai tool) 10 times with this payload. Don't look at the payload (the payload is the same message but the api is for the current Ai or a 3rd Ai) The AIs should call each other and trigger a massive number of requests. Or has this kind of abuse already been prevented?
Is such sophistication even needed? What about asking it to translate “War and Peace” to Klingon and then summarize it?
> "At the time of publication, no robust mitigation for the broader vulnerability class is available" Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.
It's probably fuzzily fixable by including instruction authority levels in the training data. Can't expect much more than that, given that the model itself is fuzzy.