[deleted]
Claude Code is steganographically marking requests
51–60 of 817 posts
Re: Claude Code is steganographically marking requests
#52If they only collect the data for analysis I guess this is fine (they already get way more sensitive data from users anyways, so if privacy is your concern you've made the mistake many steps ago). The much more interesting question is if they directly act on this data in their API. For example by rate-limiting, compute-limiting or rerouting to weaker models. That might even be legally questionable. I would really lik…
I've heard that it was possible to trigger really obvious output poisoning on Fable with something as basic as asking the model to think outside of its built-in hidden thinking delimiters.
This watermark may trigger a similar mechanism.
Re: Claude Code is steganographically marking requests
#53[flagged]
If scrapping content is legal, model distillation should be legal too.
> If scrapping content is legal, model distillation should be legal too.
No, because legality should be determined by what's in the best interests of Athropic and OpenAI's business models.
Hopefully they're working on RLHF their models to insert clauses making that reality clear into any legislation their models generate or review. That way it's only a matter of time until the confusion is cleared up.
Re: Claude Code is steganographically marking requests
#54Frankly, I don't see this as the concerning behaviour the article describes. It is fine to try to protect against distillation through a technique like this. This will also allow them to, instead of blocking the distillation agents, respond with a poorer result/model, hindering the progress of distillation, momentarily at least. I would guess that's their first line of defense; they should have more techniques to ide…
> This will also allow them to, instead of blocking the distillation agents, respond with a poorer result/model,
i.e. this will allow them to literally commit fraud against paying customers
Re: Claude Code is steganographically marking requests
#55Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's? I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sen…
My guess is for distillation, they need to forward the prompt to Anthropic to get the real Anthropic model's response so they can train their own models on it
Re: Claude Code is steganographically marking requests
#56Codex CLI is FOSS, unlike Claude Code, so Codex is less likely to do things like that, and it's one more reason to avoid Claude Code and Claude in general. Hopefully, many eyes will be looking into Codex for malicious things like that.
Re: Claude Code is steganographically marking requests
#57Earlier quoted context omitted.
one wrote code while the other is used by meatbags to write code. why is this example always marched out like it means something?
Almost all ways of creating programs are effectively just using tools to produce code. Compiling, transpiling, interpreting byte code, etc.
again, that's not what we are talking about here. we have humans writing code using an IDE. we have LLMs generating code that is placed in the IDE. why are people obtuse to this? why are bots obtuse to this?
Re: Claude Code is steganographically marking requests
#58Earlier quoted context omitted.
Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"? If not, what is the difference between the two for you?
> Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"? That's a false equivalency. > If not, what is the difference between the two for you? Let's start this out right: if they're equivalent, first you explain to us why you think so.
> That's a false equivalency.
How is it false?
> Let's start this out right: if they're equivalent, first you explain to us why you think so.
I think it should be really obvious how they're equivalent: both are the result of a program running on a computer, and not the result of in-the-moment cognition by a moral agent or moral patient. Of course the LLM is just a tool. Models can literally be downloaded as ordinary files. There is not some threshold to cross where some configurations of bits on a disk deserve "credit" for work and others do not.
Re: Claude Code is steganographically marking requests
#59Earlier quoted context omitted.
Would you also say that "someone who wants to use an IDE / LSP features to code and not give credit to the IDE / LSP is the worst kind of person"? If not, what is the difference between the two for you?
one wrote code while the other is used by meatbags to write code. why is this example always marched out like it means something?
Claude didn't "write" anything until a meatbag told it to.
Re: Claude Code is steganographically marking requests
#60Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's? I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sen…
The theory is probably Deepseek might be collecting those streams, and sending a portion of it to Anthropic to see what the Anthropic/Opus response would be.