Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

441–450 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#441

I’m surprised by the comments here being so favorable to anthropic. The comments are right about there being no “best” token, and yeah Gruber may have an agenda here. But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. Take the “gray” and “overcast” choices. And lets say before applying synthid the percentages were 48% and 52%. Those pe…

> But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. This. I want the model I'm paying for to be "pure". I don't Anthropic or anyone else messing around with it, especially not for idiotic reasons like facillitating AI stigmatization. The "safety" nonsense is obnoxious enough. They should train the best possible model and let the weigh…

You can’t be serious. The weights for composition are massively degraded by RLVR training for coding

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#443
I was using Claude yesterday and the "advice" it was giving me quickly became confused and irrelevant to the prompt, even though there was not much text in the context window. Speculation: since Claude re-reads the entire chat at every turn, the "minimal" text revisions required by watermarking quickly compound such that even Claude can't follow the discussion.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#444
This scheme has a fatal flaw, the same secret key is used for watermarking and detection. This means you need to trust that Anthropic:

* Shows the real result from their detector instead of manipulating them, you have no way to verify * Has good enough security to prevent a key leak * Rotates keys to reduce the impact of a leak (once a key is leaked anyone can rewrite text to look more/less claude generated and it becomes useless) * Will not secretly give watermark-less access to governments or high profile corporate users * Will not use multiple secret keys to track individual users. This one might be less realistic because embedding ~32 bits of signal would probably affect quality a lot more than 1 bit.

And don't forget that the detection API will work as an oracle. If it detects your content you can send it to a different model and try again until it comes back clean.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#445
post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

Anthropic had absolutely nothing to do with this. The Chinese models will soon be adopting such devices as well. It is an overwhelming force coming inter alia from educators worldwide.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#446

I’m surprised by the comments here being so favorable to anthropic. The comments are right about there being no “best” token, and yeah Gruber may have an agenda here. But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. Take the “gray” and “overcast” choices. And lets say before applying synthid the percentages were 48% and 52%. Those pe…

> Gruber may have an agenda here

Such as?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#447
> The idea that anything other than my needs should factor into the generation of text for me is patently offensive.

And this attitude is incompatible with any models produced by frontier labs. Your needs will always be subordinate to and in service of the needs of the corporation that produced the model. And we haven't even gotten to ads yet.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#448

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.

Can't the LLM just generate e.g diffs? Or some other intermediate language. Then the watermark is lost when the translation step is applied.

That's a good one. Problem is that SynthID uses only roughly the last 4 tokens, so any literal test 4 tokens or longer that is generated has the watermark. If you fix typos, insert punctuation, rename variables then yes this should be undetectable if produced from a diff.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#449
post #306

>in areas where there is an arbitrary choice between particular words or terms within the code, the watermark can be used, such as comments within code. I wonder if this is why Opus 5 keeps writing excessively long comments, even though I keep instructing it not to (both in chat, CLAUDE.md, and in its memories)

Opus 5 does not have the watermark yet. Only models released from now on.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#450

I’m surprised by the comments here being so favorable to anthropic. The comments are right about there being no “best” token, and yeah Gruber may have an agenda here. But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. Take the “gray” and “overcast” choices. And lets say before applying synthid the percentages were 48% and 52%. Those pe…

> But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. This. I want the model I'm paying for to be "pure". I don't Anthropic or anyone else messing around with it, especially not for idiotic reasons like facillitating AI stigmatization. The "safety" nonsense is obnoxious enough. They should train the best possible model and let the weigh…

> especially not for idiotic reasons like facillitating AI stigmatization.

From the people I’ve talked to at universities, LLM based cheating in education is an unstoppable nightmare. I don’t have a problem with LLMs. But I do want the cheating to - somehow - stop. The people who cheat miss out on learning. And the people who don’t cheat have their degrees devalued by those who do cheat.

Post reply on HN