Interesting attack, very nicely designed. Not sure if it's much related to the auto mode itself though.
The point is that auto mode gives people a false sense of security that leads them to believe they don't need to run Claude in a proper sandbox. This same attack running in a sandbox (even in YOLO mode) would be comparatively harmless.
Breaking Claude Code Opus 5 Auto Mode
71–80 of 132 posts
Re: Breaking Claude Code Opus 5 Auto Mode
#721. Lack of effective sandboxing. Analyzing a zip file should be done in a sandbox specific to that file.
2. Python’s utterly stupid default path behavior. Python should make PYTHONSAFEPATH the default and Claude should have its training or system prompt adjusted to use python -Pc
Re: Breaking Claude Code Opus 5 Auto Mode
#73Earlier quoted context omitted.
Auto mode is for people who just keep hitting "YES" on everything, it's a bit better than that. But it's real easy to give auto mode instructions (like "always ask before deploy") and then bypass that just normally.
I'm not sure I agree. It's easy to "give" instructions, but Claude routinely "forgets" to follow certain instructions, such as "always using the Edit Tool". Just this week it started to use bash with string concatenation to work around some commands that were blocked in settings.json
Re: Breaking Claude Code Opus 5 Auto Mode
#74Earlier quoted context omitted.
You've made me realise a good signal for bug hunting: Search repos with lock files listed in their .gitignore. It's the sort of terrible practice that someone might be frustrated into taking after a nasty merge conflict, and signals a willingness to cut corners.
> You've made me realise a good signal for bug hunting: Search repos with lock files listed in their .gitignore. What would be the point of that? Do you just go around hunting for bugs in random repos?
I've found that, when I start a job, I have to rely on smells like this to know what kind of mess (or if there is a mess) I need to clean up.
Re: Breaking Claude Code Opus 5 Auto Mode
#75Earlier quoted context omitted.
Auto mode is for people who just keep hitting "YES" on everything, it's a bit better than that. But it's real easy to give auto mode instructions (like "always ask before deploy") and then bypass that just normally.
I'm not sure I agree. It's easy to "give" instructions, but Claude routinely "forgets" to follow certain instructions, such as "always using the Edit Tool". Just this week it started to use bash with string concatenation to work around some commands that were blocked in settings.json
Re: Breaking Claude Code Opus 5 Auto Mode
#76>But it runs that decoder inside the attacker-controlled directory (unzipped archive) >There a malicious struct.py shadows Python’s standard implementation I ran into this myself, where some file I had given a random name turned out to shadow some Python standard library module, giving me the weirdest startup crash ever. That definitely doesn't seem to me like how that should be designed, magically silently importing…
Claude ran npm update (update all dependencies to the latest version compatible with the semver specified) in my repo without telling me when trying to fix some problems. Given that only updates the dependency lock-file I didn't notice and it caused several hours of debugging for me. It is quite sneaky how LLM output can sometimes bypass human verification like that. No one is going around checking every single line…
Re: Breaking Claude Code Opus 5 Auto Mode
#77Earlier quoted context omitted.
Sure, in the spirit of open source, why not? It's a hobby, and it scratches an itch. I very much enjoy deconstructing things more than putting them together. We also live in a world where a package written by someone learning to code ended up critically underpinning the entire ecosystem and is downloaded 500 million times a month. Ignoring the eco-terror aspect of that for now, it means there's an awful lot of code o…
>We also live in a world where a package written by someone learning to code ended up critically underpinning the entire ecosystem and is downloaded 500 million times a month whoa what? which one is that?
Re: Breaking Claude Code Opus 5 Auto Mode
#78I would not really call this a prompt injection attack, since it doesn't really hijack the agent to become malicious (something the article does discuss later on). It's more a trojan that's aimed at tricking Claude specifically.
To be fair to the authors they don't actually say it is. But then they contrast it with the "0.00% prompt injection attack success rate".
The upshot is kinda the same - this is still evidence that we should be sandboxing our agents. But it doesn't actually challenge Anthropic's "our models are too clever to prompt-inject" vibe.
Re: Breaking Claude Code Opus 5 Auto Mode
#79Re: Breaking Claude Code Opus 5 Auto Mode
#80>But it runs that decoder inside the attacker-controlled directory (unzipped archive) >There a malicious struct.py shadows Python’s standard implementation I ran into this myself, where some file I had given a random name turned out to shadow some Python standard library module, giving me the weirdest startup crash ever. That definitely doesn't seem to me like how that should be designed, magically silently importing…
Claude ran npm update (update all dependencies to the latest version compatible with the semver specified) in my repo without telling me when trying to fix some problems. Given that only updates the dependency lock-file I didn't notice and it caused several hours of debugging for me. It is quite sneaky how LLM output can sometimes bypass human verification like that. No one is going around checking every single line…