Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

141–150 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#141

Post by beering »

pessimizer wrote: post #87
Art9681 wrote: post #73

Earlier quoted context omitted.

If this is true then the probability of the detection tools flagging completely human generated text as AI generated is non-trivial. Let's say I write a completely original piece and the detection tool says there is a 36% probability it was generated with Claude. What then? Now it's up to the person looking at the score to cast a subjective judgement. Maybe to me, anything over 25% is unacceptable. Maybe to someone e…

I don't think that's true. I think it's a binary 0% or near 100% probability of a watermark having been detected; the more changes to the text having been made after the text was output by the LLM and the less leeway the LLM had for probable word choices, the longer the passage necessary to see it. The "problem" is that seeing the watermark doesn't mean that the person claiming to be the author didn't make extensive…

No, watermark detection is not binary, you get a real number. You decide on a threshold when looking for the watermark. This is the problem - by random chance, some human text will be detected as watermarked. You can turn the detection threshold up until it guarantees <0.001 false positive rate at the expense of higher false negatives, but seems inevitable that someone gets wrongly flagged.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#142

Post by pizzly »

inigyou wrote: post #126
ButlerianJihad wrote: post #22

Earlier quoted context omitted.

That's an intriguing twist, isn't it? It could lead to a tug-of-war. Working backwards: if it is possible to confirm 100% confidence that a chunk of text is LLM output, then it is "PD until proven otherwise". How can a human reliably assert human authorship of their source text? When all watermark tests fail? Is that proof of humanity now? If a human proves human authorship, and LLM watermarking tests positive, then…

How do you assert it now? I post some text on the internet, you claim you have copyright, how do you prove that?

Some camera pointing at you while you work. Guessing the camera needs to be watermarked itself, which can be done by adding some watermark on the chip level (each camera will have different watermark), manufacturer can confirm the watermark.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#144

Post by huflungdung »

inigyou wrote: post #108
brookst wrote: post #68

Earlier quoted context omitted.

This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.

No, they really do one at a time. You're incorrect on that. Mathematically, a long chain of conditional probabilities is equivalent to a single probability over the whole range. But computationally, for that to work out, the computation for the first probability needs to somehow consider all the downstream probabilities depending on it, which obviously isn't how autoregressive language models work. They can pack in a…

[dead]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#145

Post by beering »

dofm wrote: post #82
brookst wrote: post #74

Earlier quoted context omitted.

It didn’t take, apparently.

It’s fully possible I didn’t explain it very well in the first place, but he is making a wider point. The point I made (quite briefly) is that watermarking is only feasible because for good writing it is necessary to use T>0, or the writing will never explore a more creative choice, and that at T=0 you don’t even need a watermark to spot LLM-generated text. The point he is making is consistent with this, isn’t it? Ei…

No, you are jumping to conclusions about how watermarking works. This is some audiophile thinking that because your RNG is “pure”, you get text with an expansive soundstage or whatever. Intuitively this may be true or false depending on your personal prior but you’d need to show it mathematically. The overall token distribution shouldn’t change and the frequency at which you see the word “load-bearing” will remain the same.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#146

Post by beering »

smb06 wrote: post #117
smallerize wrote: post #7

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.

The thing that could change is interpreting "the whole thing as generated by Claude"

It’s a strawman argument because if the LLM is really just “proofreading” for you, there will be little or no watermarked text in your writing. Not enough to trip the watermark detector.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#147

Post by [deleted] »

bagacrap wrote: post #140

> But only Anthropic will be able to determine if text was seemingly generated by Claude, and Anthropic will only be able to detect the watermarks that are applied by Claude. Claude can’t detect the hidden watermark signals generated by, say, Gemini, and Gemini can’t detect the hidden watermark signals created by Claude, because each implementation is predicated on secret keys held only by the LLM provider Well, aksh…

[deleted]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#148

Post by [deleted] »

beering wrote: post #128
dragonwriter wrote: post #15

Earlier quoted context omitted.

That's inaccurate in two ways: (1) The behavior that is approximately what you describe is not "fundamental" (though it may not be something you can disable on some hosted providers), it is an option that is not fundamental (and with runtimes where you have full control can be either disabled or tuned in a large number of manners), and (2) The actual behavior that is approximately what you describe already usually in…

> which inherently compromises quality. I don’t see how this follows? Tokens are chosen randomly. If you choose tokens with a different RNG in the same distribution, you’re still getting equally good or bad tokens.

[deleted]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#150

Post by inigyou »

pizzly wrote: post #142
inigyou wrote: post #126

Earlier quoted context omitted.

How do you assert it now? I post some text on the internet, you claim you have copyright, how do you prove that?

Some camera pointing at you while you work. Guessing the camera needs to be watermarked itself, which can be done by adding some watermark on the chip level (each camera will have different watermark), manufacturer can confirm the watermark.

Is that realistically how that is proven in court today?
Post reply on HN