Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

21–30 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#21

Post by arjie »

It seems fine. I use an LLM to argue with me prior to posting blog posts so that I don't post obvious incorrectness, but the UX element to it is that it constructs notes about various sections of the text and we talk about those. There's no way for the generated text to enter the blog unless I copy-paste it and I'm not going to do that because the entire point is for me to write it.

At the point that you're generating entire volumes of text from Claude you're not really trying to be a sophisticated writer. I don't see how it's going to hurt for it to choose random related words.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#22

Post by ButlerianJihad »

fwipsy wrote: post #12
ButlerianJihad wrote: post #10

Earlier quoted context omitted.

It is quite just, if you think about it. Human works are copyrighted and protected at the moment of creation. All rights reserved. Yet, LLM outputs are uncopyrightable. Therefore, if Claude or any AI has processed my copyrighted work, the end result is uncopyrightable and in the Public Domain. The public has a right to know: is this a human copyrighted work, an LLM PD work, or is the human falsely claiming authorship…

Perhaps LLM outputs are uncopyrightable, but derivative works of copyrighted works are not automatically in the public domain.

That's an intriguing twist, isn't it? It could lead to a tug-of-war.

Working backwards: if it is possible to confirm 100% confidence that a chunk of text is LLM output, then it is "PD until proven otherwise". How can a human reliably assert human authorship of their source text? When all watermark tests fail? Is that proof of humanity now?

If a human proves human authorship, and LLM watermarking tests positive, then is that going to be considered a "derivative work" or not? What if there is an applicable license for the source work, such as "CC-BY-ND" that prohibits derivative works?

This has not been court-tested, and I expect that it will need testing at that level before we can have any assurances.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#23

Post by smallerize »

andy99 wrote: post #8
smallerize wrote: post #6

Earlier quoted context omitted.

Yes, and it says that in https://www.anthropic.com/news/claude-text-watermark

I should have read that, it’s actually quite reasonable and I don’t really understand the objections in TFA having read it. > One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter.…

I think Anthropic should have put all the info into one blog post. Splitting it up is really confusing people.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#24

Post by jleyank »

smallerize wrote: post #7

Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.

Rands made this point a few days ago as I recall. Worries about having his tool corrupt his writing during editing, etc.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#25

Post by Finnucane »

"Anthropic's . . . Claude is a Perversion of Writing."

FITFY.

I have no sympathy for writers whining about what the AI is doing to 'their' writing. It's only your writing when you write it. There's any easy way to avoid this: don't fucking use it. Use you own brain.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#26

Post by levocardia »

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#27

Post by scuppernong »

npilk wrote: post #19
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…

auto-oulipo

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#28

Post by reliablereason »

dragonwriter wrote: post #15
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

That's inaccurate in two ways: (1) The behavior that is approximately what you describe is not "fundamental" (though it may not be something you can disable on some hosted providers), it is an option that is not fundamental (and with runtimes where you have full control can be either disabled or tuned in a large number of manners), and (2) The actual behavior that is approximately what you describe already usually in…

(1) LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time. This is a consequence of how they are trained.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#29

Post by lemarchr »

Some here are arguing that mechanisms used by LLM providers already derail the goal of "the very best, most precise words at every single decision point", therefore the author is misguided.

The author has expressed a preference. Assume that there is a sequence of tokens, such that it is considered the absolute best by the author. This particular method of watermarking makes it less likely to generate that sequence, by definition.

I feel their argument would have been clearer and stronger if they had spent more time exploring the alternatives, and whether these alternatives would be just as effective. It is trivially easy to remove invisible tokens.

Like it or not, there is a public good to being able to identify AI generated content, and a small degredation in quality is tolerable in my opinion.

I don't think anybody has to worry about this issue though. Manual writing, coding, and proof reading continues to be an option. Where AI output is nothing to be ashamed of, the tools are available. For everyone else, there will be LLM providers that ignore EU law.

Post reply on HN