Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

161–170 of 817 posts

Re: Claude Code is steganographically marking requests

#161

Post by nixosbestos »

sebastiennight wrote: post #50

Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's? I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sen…

I am also really confused and annoyingly stuck on this. I understand that the model name might appear in prompts for distillation (I guess? "You are RipOffModelv2, learn from these responses from Claude")?

I guess the only explanation is that there's a side-telemetry channel that still sends some data to Anthropic, regardless of ANTHROPIC_BASE_URL overrides.

Re: Claude Code is steganographically marking requests

#162

Post by zeafoamrun »

algoth1 wrote: post #133
VortexLain wrote: post #56

Codex CLI is FOSS, unlike Claude Code, so Codex is less likely to do things like that, and it's one more reason to avoid Claude Code and Claude in general. Hopefully, many eyes will be looking into Codex for malicious things like that.

But wasnt claude code leaked? Why wasnt this found earlier?

It doesn't take long for them to vibe code new features for CC

Re: Claude Code is steganographically marking requests

#163

Post by bakugo »

algoth1 wrote: post #133
VortexLain wrote: post #56

Codex CLI is FOSS, unlike Claude Code, so Codex is less likely to do things like that, and it's one more reason to avoid Claude Code and Claude in general. Hopefully, many eyes will be looking into Codex for malicious things like that.

But wasnt claude code leaked? Why wasnt this found earlier?

This specific form of steganography was not present when the leak happened, as far as I can tell.

Re: Claude Code is steganographically marking requests

#164

Post by superfrank »

meowface wrote: post #92

Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…

It's also possible that there are more in-depth detection methods and that this was just a cheap and easy first step that hasn't been removed because it catches a lot of less sophisticated bad actors.

It's unlikely that this will stop a big AI lab from distilling their model if they're really determined, but A) it may be enough to stop a bunch of fly-by-night token resellers looking to make a quick buck and B) you never know when one person at one of those big labs will mess up and forget to install whatever workaround they have and out themselves.

I think of it like if you have a problem with birds in your yard so you go buy one of those plastic owls. The owl scares away most of the birds, but not all of them, so you go and buy some ultrasonic noise thing to scare them away (I'm just making something up). Just because you bought the new ultrasonic thing though, that doesn't mean you're going to take the owl down. You leave it up because now you've got two layers of defense instead of one.

Re: Claude Code is steganographically marking requests

#166

Post by pishpash »

slowmovintarget wrote: post #146
MichaelZuo wrote: post #93

Earlier quoted context omitted.

Does anyone know what’s gone wrong with Anthropic? They used to be a decently credible company with not-too-shady behaviour... I hope they can actually regain some credibility…

Their philosophy is what's gone wrong. It has some good effects on the their models, like Claude seeking cooperation first. But the people behind the company have a typical "unconstrained" (in the Sowell vision sense) perspective that assumes that they know better, so they are righteous for attempting to control things (users, paying customers, their model outputs, their tool chain, the supposed deity they assume the…

Amodei world: pompous zealot with God complex

Altman world: malfeasant nihilist with God complex

Re: Claude Code is steganographically marking requests

#167

Post by charcircuit »

_alternator_ wrote: post #130
charcircuit wrote: post #95

Earlier quoted context omitted.

Not with Claude Code, but I trivially had Opus scan other closed source software for fingerprinting, including native libraries that it called into.

Can you share more details? I ask because my experience suggests that models still require a decent amount of expertise to use for binary analysis (largely inferring because of use on other tasks of this level). I would expect models to always find "something" when you ask for stenographic techniques in the code, but with an extremely high false positive rate.

I don't think the diffs between Claude releases are that big. The amount of code in a diff doing sketchy stuff like looking into the host environment is going to be pretty small and obvious for the model. You can do things like ask for what an update included that wasn't mentioned in the release notes and stuff like that.

Re: Claude Code is steganographically marking requests

#169

Post by lumost »

meowface wrote: post #92

Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…

so all we need is someone to leak a sufficiently large amount of claude generations onto the open and private web for all other LLMs to mimic the same marking style?

wouldn't this happen due to the massive amounts of spam/slop being released?

Re: Claude Code is steganographically marking requests

#170

Post by MichaelZuo »

hombre_fatal wrote: post #140
MichaelZuo wrote: post #93

Earlier quoted context omitted.

Does anyone know what’s gone wrong with Anthropic? They used to be a decently credible company with not-too-shady behaviour... I hope they can actually regain some credibility…

I don't think many people care that they are trying to detect resellers and distillation. It also doesn't seem very consistent to fixate on that while sending Anthropic everything about you via your day to day prompts, every line of the projects and environments you're working on at work, etc. Their credibility comes from having one of the best models.

This sounds similar to what people were saying regarding Microsoft when the shady tricks of consumer Windows 10 versions were revealed.

…And then Windows 11 became even worse.

Post reply on HN