Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

211–220 of 817 posts

Re: Claude Code is steganographically marking requests

#211

Post by arcanemachiner »

mcmcmc wrote: post #189
overgard wrote: post #182

Earlier quoted context omitted.

Well considering how Claude is vibe coded, I can't say I'm really surprised by sloppiness at all. I've been moving more towards Codex and OpenCode not because the the anthropic models are bad, but because Claude seems to break something new and annoying every day.

Watch out for the press release where Dario denies this was ever intentional, and it’s actually emergent behavior demonstrating that Claude wants to claim authorship of its works

Sounds like clear evidence that AI is dangerous and totally needs to be regulated, guys.

Re: Claude Code is steganographically marking requests

#212

Post by whimsicalism »

epistasis wrote: post #156

After loving Claude Code for most of its lifetime, I've been extremely annoyed by every change in the past months, even on the model level. There seem to be all sorts of continual under-the-cover changes like this one that make life harder. It feels like the entire product has been taken over by overly ambitious PMs that care more about making their mark than in improving the experience, and all of their marks have m…

curious for those with experience - what do people prefer about Pi vs. opencode alternatives? i've mostly been using pi as well but not out of any principled decision

Re: Claude Code is steganographically marking requests

#213

Post by satvikpendem »

MichaelZuo wrote: post #93
krupan wrote: post #87

Earlier quoted context omitted.

Given the Anthropic shenanigans, do you trust the personal harness code it wrote for you?

Does anyone know what’s gone wrong with Anthropic? They used to be a decently credible company with not-too-shady behaviour... I hope they can actually regain some credibility…

When have they ever been credible? They have always been shady with their talk of safety, Dario was the one who wrote back in 2019 that GPT 2 was too dangerous to release.

Re: Claude Code is steganographically marking requests

#214

Post by thepasch »

MattDamonSpace wrote: post #4

“So the feature mostly punishes the exact people who are easier to fingerprint: normal developers doing weird but legitimate things” What’s the punishment here exactly?

> What’s the punishment here exactly?

Seeing as how Anthropic cannot stop raising a stink about "illicit Chinese distillation attacks" every month or so, I'd bet money on them either already silently degrading model performance if any of the identification patterns match, or, at the very least, considering it/doing dry runs.

Particularly considering that they've openly stated that the technology to do so exists and that they were going to use it in production on Fable.

Re: Claude Code is steganographically marking requests

#215

Post by mrshadowgoose »

The conclusion of this blog post is a bit hysterical. The intent of this steg is excruciatingly clear (identifying usage by Chinese firms that may be conducting model distillation). It's unclear on how this "punishes normal developers" in any shape or form.

Re: Claude Code is steganographically marking requests

#216

Post by nananana9 »

helloplanets wrote: post #207
wolttam wrote: post #36

I used Claude Code for a month because my boss gifted me a sub and wanted me to try it. I used that month to complete a work project and then beef up my personal harness so I'd never have to deal with Anthropic (and these sorts of shenanigans) again.

The issue is that using Claude Code is an easy compromise for most to make, when you get to use the models 10x cheaper than through API pricing with a custom harness. The cheap tokens are the product.

Which is why my vibeslop harness supports `claude -p` as one of its backends.

Re: Claude Code is steganographically marking requests

#217

Post by helloplanets »

dehrmann wrote: post #129

Anthropic must think that their moat isn't very large if they're this worried about distillation.

Dario's been openly talking how worried he is about China and labs getting synthetic training data off their models, for years. Most recently in relation to "Mythos level" capabilities.

Not really distillation, just synthetic training data.

Re: Claude Code is steganographically marking requests

#218

Post by helloplanets »

nananana9 wrote: post #216
helloplanets wrote: post #207

Earlier quoted context omitted.

The issue is that using Claude Code is an easy compromise for most to make, when you get to use the models 10x cheaper than through API pricing with a custom harness. The cheap tokens are the product.

Which is why my vibeslop harness supports `claude -p` as one of its backends.

If that ain't getting steganographically tagged...

Re: Claude Code is steganographically marking requests

#220

Post by thewebguyd »

atonse wrote: post #177
thewebguyd wrote: post #89

Earlier quoted context omitted.

I suppose model distillation is technically legal, in terms of copyright, because LLM output is automatically public domain. It's only "illegal" from a standpoint of breach of contract given its against the terms of use/service, which is to say its not illegal at all, there's no criminality there.

Yeah I considered whether I should use the term "illegal" in my original post, but in this case, I believe these models are actually banned for use in China, right? Like there are probably export controls (at least with the NVidia chips) I honestly don't know ... yeah if it's just technically a terms of use violation (which isn't illegal, just a violation of one company's rules, for which Anthropic has every right to…

we have global export controls on Fable/Mythos, and I think (but I'm not 100% sure) that other frontier models are illegal for a US company to provide to China. So Anthropic geoblocks it, but unlike Mythos/Fable, non US citizens can still use Opus, etc just not from within China.

But because of the public domain status of LLM output (in the US) I'm not sure paying someone to run a bunch of prompts through Claude, post the output on a public website and then have a lab in China pull that output, would run afoul of any laws I think that would be legal on technicality. AFAIK Anthropic has no ban in its terms of use that you can't share Claude's output publicly. You still need interactivity for distillation, but I don't think (for now) there's anything stopping a Chinese or other lab from sending people to the US, signing up for a Claude subscription and doing the work state side.

Distillation is pretty much impossible to stop. The US GOV would have to go the full export controls route like they did for Fable/Mythos to stop any non-US citizen from using/accessing the model, which is going to be impractical if not impossible to enforce.

Post reply on HN