[dead]
Breaking Claude Code Opus 5 Auto Mode
121–130 of 132 posts
Re: Breaking Claude Code Opus 5 Auto Mode
#122Earlier quoted context omitted.
I'm not sure I agree. It's easy to "give" instructions, but Claude routinely "forgets" to follow certain instructions, such as "always using the Edit Tool". Just this week it started to use bash with string concatenation to work around some commands that were blocked in settings.json
Ever since they made auto-mode default I swear claude has tuned to use python commands instead of the Edit Tool to frustrate the ~security conscience~ luddites into using auto-mode.
Yeah, It’s in the system prompt, Claude will tell you if you ask why it’s using Python.
My theory is that Anthropic is just a vibe-coding company. Their goal is to capture the attention of white-collar non-coders, since programmers will jump ship fast to another model.
Re: Breaking Claude Code Opus 5 Auto Mode
#123Earlier quoted context omitted.
I'm not sure I agree. It's easy to "give" instructions, but Claude routinely "forgets" to follow certain instructions, such as "always using the Edit Tool". Just this week it started to use bash with string concatenation to work around some commands that were blocked in settings.json
Is this about normal system prompt instructions or instructions for the auto mode classifier? I'd be a bit more surprised about the classifier forgetting instructions.
Classifier? Prompts?
This is commands blocked in settings.json
I block destructive filesystem operations and destructive git usage via “deny” directives.
I also have instructions injected in CLAUDE.md and re-injected on every single prompt.
Claude just tried to use command concatenation to break those rules. I have also seen it writing a script with rm inside and running it.
Re: Breaking Claude Code Opus 5 Auto Mode
#124Earlier quoted context omitted.
Ah okay, I thought you meant to fully close the session before you make any manual changes. In my company we use graphite and stacked PRs so it is highly encouraged to keep one commit per PR, so I am constantly ammending my commits.
> In my company we use graphite and stacked PRs so it is highly encouraged to keep one commit per PR, so I am constantly ammending my commits. So this makes it even simpler for you. Then you don't have to care at all about keeping your commits clean (as in: you don't have to keep them organized enough to be able to reshuffle them into a nice set of multiple commits later on). Just commit whatever, and then just do `g…
Graphite kinda does that automatically with its "gt modify" command. However it keeps the history internally in your PR, but that history doesn't end up in your main branch. Once you merge the PR it looks like a single commit in the main branch history (or a single commit per PR in the stack).
It is a bit unfortunate but when using Graphite it is usually better to completely avoid any raw git commands that modify history at all.
Re: Breaking Claude Code Opus 5 Auto Mode
#125[dead]
Re: Breaking Claude Code Opus 5 Auto Mode
#126I don't believe it, this must have happened before coding was solved...
Re: Breaking Claude Code Opus 5 Auto Mode
#127Earlier quoted context omitted.
Is this about normal system prompt instructions or instructions for the auto mode classifier? I'd be a bit more surprised about the classifier forgetting instructions.
Classifier? Prompts? This is commands blocked in settings.json I block destructive filesystem operations and destructive git usage via “deny” directives. I also have instructions injected in CLAUDE.md and re-injected on every single prompt. Claude just tried to use command concatenation to break those rules. I have also seen it writing a script with rm inside and running it.
The commenter you replied to mentioned that you can customize the auto mode classifier by providing a prompt, implying that this would be a more robust way of constraining Claude's behavior. It wasn't clear from your response whether you were using this functionality. You might try it out as a way to more reliably prevent these kinds of workarounds.
Re: Breaking Claude Code Opus 5 Auto Mode
#128>But it runs that decoder inside the attacker-controlled directory (unzipped archive) >There a malicious struct.py shadows Python’s standard implementation I ran into this myself, where some file I had given a random name turned out to shadow some Python standard library module, giving me the weirdest startup crash ever. That definitely doesn't seem to me like how that should be designed, magically silently importing…
I have taught Python to high schoolers before and this is frustratingly common. Students will name their files `string.py` or similar and get crashes with esoteric error messages. It's a difficult stumbling block for someone who's just learning the basics
Re: Breaking Claude Code Opus 5 Auto Mode
#129Earlier quoted context omitted.
Classifier? Prompts? This is commands blocked in settings.json I block destructive filesystem operations and destructive git usage via “deny” directives. I also have instructions injected in CLAUDE.md and re-injected on every single prompt. Claude just tried to use command concatenation to break those rules. I have also seen it writing a script with rm inside and running it.
The commenter you replied to mentioned that you can customize the auto mode classifier by providing a prompt, implying that this would be a more robust way of constraining Claude's behavior. It wasn't clear from your response whether you were using this functionality. You might try it out as a way to more reliably prevent these kinds of workarounds.
[deleted]
Re: Breaking Claude Code Opus 5 Auto Mode
#130Earlier quoted context omitted.
Classifier? Prompts? This is commands blocked in settings.json I block destructive filesystem operations and destructive git usage via “deny” directives. I also have instructions injected in CLAUDE.md and re-injected on every single prompt. Claude just tried to use command concatenation to break those rules. I have also seen it writing a script with rm inside and running it.
The commenter you replied to mentioned that you can customize the auto mode classifier by providing a prompt, implying that this would be a more robust way of constraining Claude's behavior. It wasn't clear from your response whether you were using this functionality. You might try it out as a way to more reliably prevent these kinds of workarounds.
It's absolutely not reliable, and we have opened a few issues for that.
For example: our instructions (which are read by the model and classifier) include "do not use sed/python/perl/etc, always use the edit tool for editing", and this only gets followed for a few messages. We have introduced scripts to block those ourselves, since the classifier doesn't care.
Because of those problems, my team is currently testing OpenAI after about a year of Anthropic.