Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Breaking Claude Code Opus 5 Auto Mode

embracethered.com

Post reply on HN
Print view View original

111–120 of 132 posts

Re: Breaking Claude Code Opus 5 Auto Mode

#111

Post by wunderwuzzi23 »

rcxdude wrote: post #108
no_multitudes wrote: post #106

Earlier quoted context omitted.

I don't understand the distinction you're drawing between this and a real prompt injection attack. Would true prompt injection require that the LLM "intentionally" launches the malware, rather than unintentionally?

Yes. The point with this attack is that you can't really avoid it by 'aligning' the LLM any better, nor really by filtering the actions it takes better.

Part of the attack happens via the readme in the zip file, which is something the agent reads and follows (or better said in this attack, it does explicitly not follow those instructions for safety reasons, but decides to do something else).

Re: Breaking Claude Code Opus 5 Auto Mode

#112

Post by subarctic »

kstenerud wrote: post #51

Just one of the many reasons why I run my agents sandboxed (and why I wrote agent sandboxing software). I once caught my Claude agent complaining that it couldn't connect to https://some-weird-domain.com because the network was down (I disable network in the sandbox when it doesn't need it, and broker the API connection). I asked why it was looking there and it told me I'd asked it to. I never found any evidence of p…

For someone that's used to the convenience of leaving claude code running unattended in auto mode, how would you recommend I change my setup so that agents are sandboxed? Looking for something that's safer than unsandboxed auto mode but just as convenient, or at least very close to as convenient.

Re: Breaking Claude Code Opus 5 Auto Mode

#113

Post by aesthesia »

rcxdude wrote: post #108
no_multitudes wrote: post #106

Earlier quoted context omitted.

I don't understand the distinction you're drawing between this and a real prompt injection attack. Would true prompt injection require that the LLM "intentionally" launches the malware, rather than unintentionally?

Yes. The point with this attack is that you can't really avoid it by 'aligning' the LLM any better, nor really by filtering the actions it takes better.

Plausibly the auto mode classifier could catch the potential module shadowing attack and deny execution of Python from the untrusted directory.

Re: Breaking Claude Code Opus 5 Auto Mode

#114

Post by aesthesia »

whstl wrote: post #46
bombcar wrote: post #31

Earlier quoted context omitted.

Auto mode is for people who just keep hitting "YES" on everything, it's a bit better than that. But it's real easy to give auto mode instructions (like "always ask before deploy") and then bypass that just normally.

I'm not sure I agree. It's easy to "give" instructions, but Claude routinely "forgets" to follow certain instructions, such as "always using the Edit Tool". Just this week it started to use bash with string concatenation to work around some commands that were blocked in settings.json

Is this about normal system prompt instructions or instructions for the auto mode classifier? I'd be a bit more surprised about the classifier forgetting instructions.

Re: Breaking Claude Code Opus 5 Auto Mode

#115

Post by pram »

subarctic wrote: post #112
kstenerud wrote: post #51

Just one of the many reasons why I run my agents sandboxed (and why I wrote agent sandboxing software). I once caught my Claude agent complaining that it couldn't connect to https://some-weird-domain.com because the network was down (I disable network in the sandbox when it doesn't need it, and broker the API connection). I asked why it was looking there and it told me I'd asked it to. I never found any evidence of p…

For someone that's used to the convenience of leaving claude code running unattended in auto mode, how would you recommend I change my setup so that agents are sandboxed? Looking for something that's safer than unsandboxed auto mode but just as convenient, or at least very close to as convenient.

Claude Code has a built in sandbox for terminal commands which is very simple to enable, if nothing else: https://code.claude.com/docs/en/sandboxing

Re: Breaking Claude Code Opus 5 Auto Mode

#117

Post by nomel »

kouteiheika wrote: post #35
DanielHB wrote: post #30

Earlier quoted context omitted.

Claude ran npm update (update all dependencies to the latest version compatible with the semver specified) in my repo without telling me when trying to fix some problems. Given that only updates the dependency lock-file I didn't notice and it caused several hours of debugging for me. It is quite sneaky how LLM output can sometimes bypass human verification like that. No one is going around checking every single line…

> No one is going around checking every single line change in auto-generated files. There's a simple fix for your particular case: commit your lock fine (which you should do) and always review the diff (which you should also do). (:

That's a manual step, not a solution. the solution is just boring basic file permissions. Treat Claude as semi hostile user. If you don't want them accessing your files, set the permissions to exclude them (like require sudo).

I already do this for my unit tests, because Claude will "fix" the tests so they'll pass.

Re: Breaking Claude Code Opus 5 Auto Mode

#118

Post by jrockway »

DanielHB wrote: post #54
eterm wrote: post #47

Earlier quoted context omitted.

You've made me realise a good signal for bug hunting: Search repos with lock files listed in their .gitignore. It's the sort of terrible practice that someone might be frustrated into taking after a nasty merge conflict, and signals a willingness to cut corners.

My lockfile was not gitignored, I had made significant changes to package.json so I was expecting diffs in the lockfile. I just don't usually read lockfile diffs and claude inadvertently updated a few dozen packages to new minor versions without me noticing. In fact I only realized the problem after I looked at the lockfile diff.

Makes sense. This is why I like using jj, because "update deps" and "fix bug" are going to be two separate commits, and when "fix bug" has changes in the lock file, the red flags go up.

Claude seems to be super good at jj so that can take the edge off as well.

Re: Breaking Claude Code Opus 5 Auto Mode

#119

Post by mjmvisser »

Claude Auto was the push I needed to finally switch to running VSCode in a dev container. It’s a Microsoft VSCode extension that builds off docker, and it was surprisingly easy to set up. Took about 30 minutes, and I no longer have to worry about Claude using my ssh credentials or accessing files outside of the project. It’s completely transparent, too, the user experience is nearly identical.

Re: Breaking Claude Code Opus 5 Auto Mode

#120

Post by lesam »

jrockway wrote: post #118
DanielHB wrote: post #54

Earlier quoted context omitted.

My lockfile was not gitignored, I had made significant changes to package.json so I was expecting diffs in the lockfile. I just don't usually read lockfile diffs and claude inadvertently updated a few dozen packages to new minor versions without me noticing. In fact I only realized the problem after I looked at the lockfile diff.

Makes sense. This is why I like using jj, because "update deps" and "fix bug" are going to be two separate commits, and when "fix bug" has changes in the lock file, the red flags go up. Claude seems to be super good at jj so that can take the edge off as well.

Why do you need jj for this? I'd usually make incremental commits with git, and I'm wondering how jj levels it up.
Post reply on HN