Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

41–50 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#41

Post by DarkmSparks »

I dont see how there would be remotely enough entropy in most model outputs for this to be close to feasible with any kind of accuracy.

Either they false positve on pretty much everything ever written, or the chances of catching a true positive is so low as to be useless.

Basically Cinavia for text, and that often falls over and is easy to remove even when there is megabytes of data streaming over a long period of time rather than 2 or 3 bits per wall of text, let alone what most people use claude for, when there is a strict dictionary and other tight output constraints.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#42

Post by skew-aberration »

smallerize wrote: post #7

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.

Can't the LLM just generate e.g diffs? Or some other intermediate language. Then the watermark is lost when the translation step is applied.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#43

Post by colmmacc »

syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

I think the article is wrong on this but it's more subtle than that. Probability distributions have a peak; there is still a token with a peak probability. What's interesting about these techniques is that token by token it can actually make the peak token even more probable. A distribution doesn't have to be "flattened" to leave a watermark - it can be "amplified" and made "more peaky".

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#45

Post by LoganDark »

I keep seeing an irritating misconception in this space, which is that the alternatives chosen by these algorithms are supposed to mean the same things as what they're displacing. That's not true, and not how LLM generation works. Complaints that two different choices don't mean the same thing miss the entire point.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#46

Post by walrus01 »

LoganDark wrote: post #40
walrus01 wrote: post #11

> I want any LLM I use to choose the very best, most precise words at every single decision point. Try running an llm like qwen 3.8 27B in Q8 locally with an intentionally very low temperature setting, it will write like a caveman crossed with a robot. You may find that an extremely literal output does not look pleasant to read for humans.

That is not what that means. Generally, precise word choice requires more than autocomplete. Larger models simulate this with hidden layers.

Excessively precise word choice does not result in something that looks like content written by, or palatable to humans. It looks like you gave a high school 12 grade student a science paper and told them to apply a thesaurus to at least one word in every sentence and replace it with something else.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#47

Post by cush »

npilk wrote: post #19
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…

Models can easily do multiple passes

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#48

Post by reader9274 »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

"Smart"? Have you read his writings in the last decade? It's all nonsense, which is why I stopped reading circa 2018

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#49

Post by avaer »

syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

That's missing the point. It's the distribution that's the "best", not the tokens. Then Anthropic comes in and makes the distribution something other than the best. The only saving grace is that Anthropic says it's not that bad.

Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that bad, and if it is they won't admit it. There's no way for anyone to check. And your argument will still be technically correct but missing the point.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#50

Post by epihelix »

smallerize wrote: post #7

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.

You know, back in the era when proofreaders were human, I never one met a proofreader who rewrote my text afresh, rather than annotating the text with a pen.

It's still possible to use Claude to proofread - highlight grammatical, flow, structure, logic errors and make simple suggestions for you to pick and choose or adapt as you wish. No watermarking will flag your text. No flaw accusations of LLM authorship will haunt you. All will be fine.

But if you want an LLM to rewrite your text, that's (a) not proofreading, and (b) should be flagged as LLM generated ... because it is.

Post reply on HN