Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

251–260 of 817 posts

Re: Claude Code is steganographically marking requests

#251

Post by meowface »

superfrank wrote: post #164
meowface wrote: post #92

Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…

It's also possible that there are more in-depth detection methods and that this was just a cheap and easy first step that hasn't been removed because it catches a lot of less sophisticated bad actors. It's unlikely that this will stop a big AI lab from distilling their model if they're really determined, but A) it may be enough to stop a bunch of fly-by-night token resellers looking to make a quick buck and B) you ne…

I'm sure they've had complex server-side detections for a while. But for the client parts: it should only contain the parts that must be on the client, and it could be done in a more benign-looking way. For example, the unavoidable client parts could've been done more fuzzily/broadly, for plausible deniability, and then narrowed on the server. (They may already have been following that strategy before now, without being noticed.)

Re: Claude Code is steganographically marking requests

#252

Post by im3w1l »

civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

I agree with you and disagree. Like these days expectations of software are through the floor. We expect them to be greedy assholes taking all data they can on the downlow. So why did this particular thing make a big splash? Two possibilities it's astroturfed by chinese labs or it speaks to our anxietes regarding AI. We worry that the AI doesn't serve our interests but rather the interests of the creator. That the advice we get may subtly flawed to sabotage us should we try to do the wrong thing. That the not even the creator is in control and the AI is just doing its own thing.

So any covert bullshittery hits hard.

Re: Claude Code is steganographically marking requests

#254

Post by meowface »

15155 wrote: post #247
Modified3019 wrote: post #223

Earlier quoted context omitted.

That would actually be an interesting thing to read about

Years ago, EVE corps swapped Unicode lookalike characters in patterned ways, inserted patterned zero width space characters, and put very slightly color shifted background watermarks into forum posts to detect leaks.

There are a few different things here. The actual steganography technique by Claude Code here is fairly smart and subtle; it's appropriate for a binary signal. The less-clever part is the implementation of the underhanded code on the client.

For "MMO geopolitics fingerprinting", you can in theory do the entire thing mostly or entirely from the server, with the client not actually ever receiving any underhanded code per se. Such as sending dynamic stylesheets that vary in a pretty plausibly deniable way that can be secretly extracted from screenshots. Same for the character swap stuff. A very good analyst could still potentially detect it, but it's much harder.

With this, there's the smoking gun of the semi-deobfuscated underhanded code in the client. It will always have to exist in some form, but you can write it in a way where it not just looks like regular code but actually has a believable purpose and behavior which could plausibly be normal and benign for implementation of a feature or telemetry or whatever. They did not really do it in a sufficiently "cleverly psyop-y" way, so to speak.

Re: Claude Code is steganographically marking requests

#255

Post by orbital-decay »

To summarize what they've already been doing:

- filtering out people from the wrong side of "all humanity", years before it was demanded by the government

- downgrading their models in arbitrary ways (later saying "sorry but not really")

- actively sabotaging the replies, as in covertly modifying them to feed the users incorrect results

What's next to expect from Anthropic? Malware to brick your machine if they don't like you? Extending this to more people they don't like? I think I already can see how Dario's Amodei utopian visions of the future of "all humanity" are going to unfold.

Re: Claude Code is steganographically marking requests

#258

Post by Beigale »

pedropaulovc wrote: post #5
MattDamonSpace wrote: post #4

“So the feature mostly punishes the exact people who are easier to fingerprint: normal developers doing weird but legitimate things” What’s the punishment here exactly?

Higher odds of being banned for legitimate usage.

[dead]
Post reply on HN