Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…
Claude Code is steganographically marking requests
571–580 of 817 posts
Re: Claude Code is steganographically marking requests
#572Earlier quoted context omitted.
Imagine an electricity generating company saying that they don't allow their electricity to be used to cold start a competitor's generator.
Do you think software should be regulated as a utility?
I do think that AI models that were trained using the biggest intellectual property heist in human history should be a utility for all, yes.
Re: Claude Code is steganographically marking requests
#573Earlier quoted context omitted.
From my understanding, distilling the model with another model is not illegal per se. Also, the output of the LLM is public domain by law, too. So, why all this "effort" to protect the model? This is a free market, and moving fast and breaking things is the norm. If they are so adamant on protecting their IP, maybe they can start by respecting others' IP, so we can start talking about ethics, equality and playing fai…
The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not, which is what China has been doing. Countless thousands of separate requests abusing the service (which is not a simple static HTML feed, but an AI service request) for every kind of query to soak up the results. The post is about what's in the local code, but fo…
And which country has "Black Sites" peppered around the world to detain and interrogate people they don't like?
Re: Claude Code is steganographically marking requests
#574Earlier quoted context omitted.
Do you think software should be regulated as a utility?
AI probably should be. The bulk of its efficacy comes from the work of “everyone else” (in loose terms). AI also aims/hope/threatens to replace such a large number and range of jobs that it probabky should be a commons.
Re: Claude Code is steganographically marking requests
#575Earlier quoted context omitted.
I mean, you’d resort to an obfuscated approach if you thought the ‘malicious’ users would remove your direct telemetry. The other users could be perfectly happy about it, but if you announced the change, obviously the malicious users would hear about it too and disable it. This isn’t a comment on whether I agree with the change. Just that your analogies aren’t applicable here.
Anthropic always came off to me as what can shorthand be described as an abusive controlling partner + the resulting relationship. If you start viewing the conversations around it through that lens, a lot of stuff intuitively clicks into place. Including this apologism for example.
One possibility: they wanted to keep it secret because they’re crooks.
Another possibility: there are so many calls that it costs a lot in operating expenses.
Remember when we mentioned that even just saying “hello” at the start of a chat costs extra money for no reason?
So if I create a mechanism on the client side that every transmission uses 4 bytes instead of 40… it is clever way how to spend less.
Of course, it doesn’t excuse them for not mentioning it anywhere… or for hiding it on page 385 of the terms of service… like when the Vogons told Earthlings that the notice about Transgalactic Highway was in the basement on Alpha Centauri :)
Re: Claude Code is steganographically marking requests
#576Earlier quoted context omitted.
Anthropic always came off to me as what can shorthand be described as an abusive controlling partner + the resulting relationship. If you start viewing the conversations around it through that lens, a lot of stuff intuitively clicks into place. Including this apologism for example.
Why would they want to do that when it’s likely that someone will find out anyway and it could turn into a scandal? One possibility: they wanted to keep it secret because they’re crooks. Another possibility: there are so many calls that it costs a lot in operating expenses. Remember when we mentioned that even just saying “hello” at the start of a chat costs extra money for no reason? So if I create a mechanism on th…
Things do not need to make sense. They often do not. They just appear just enough like they would so that it flies under the radar.
It's all just conway's law. It had to be like this. It cannot be any other way.
Re: Claude Code is steganographically marking requests
#577Earlier quoted context omitted.
I don’t mean to insult you, but it is probably worth refreshing yourself on when slippery slope is actually a fallacy. It’s not correct to suggest logical extension of an established precedent is a slippery slope.
Putting "I don't mean to insult you" before a mild phrase just makes it sound you actually want to insult but in a passive aggressive way.
Re: Claude Code is steganographically marking requests
#578Re: Claude Code is steganographically marking requests
#579The conclusion of this blog post is a bit hysterical. The intent of this steg is excruciatingly clear (identifying usage by Chinese firms that may be conducting model distillation). It's unclear on how this "punishes normal developers" in any shape or form.
Tons of normal developers use ANTHROPIC_BASE_URL, the flag which activates the malware.
Re: Claude Code is steganographically marking requests
#580Earlier quoted context omitted.
From my understanding, distilling the model with another model is not illegal per se. Also, the output of the LLM is public domain by law, too. So, why all this "effort" to protect the model? This is a free market, and moving fast and breaking things is the norm. If they are so adamant on protecting their IP, maybe they can start by respecting others' IP, so we can start talking about ethics, equality and playing fai…
The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not, which is what China has been doing. Countless thousands of separate requests abusing the service (which is not a simple static HTML feed, but an AI service request) for every kind of query to soak up the results. The post is about what's in the local code, but fo…
This is literally what the "training AI on copyrighted works is just like a human learning/getting inspired" crowd has been arguing though.
Literally. People have been literally saying that it was wrong because they did this "learning" at scale in a dishonest way.