Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

261–270 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#261

Post by vrganj »

TheOtherHobbes wrote: post #238

The issue here isn't (just) adulteration, it's that watermarking in general is unworkable. If all the providers use watermarking systems with different shifting logit weightings, and the keys are secret, you have to check every provider to see if it produced a given text. Which is clearly ridiculous. And if all providers collaborate and use the same weightings, or if the weightings are constant and not rotated crypto…

> Interoperability. Providers must implement an interoperability solution for watermark detection such as a standardized API access method, a publicly readable signpost mechanism embedded in content, or participation in a consortium detection solution by February 2, 2027

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#263

Post by fwlr »

If we wish to use “the very best, most precise words at every single decision point” then I will note that “writing” is not the best word for what LLMs are doing. A more precise synonym might be “generating”, in which case this essay becomes “Watermarking is a perversion of text generation” - which is true, albeit somewhat trivial.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#264

Post by regularfry »

simonh wrote: post #239
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

The quality of the LLM just _is_ the quality of the token probabilities it generates. Better quality token probabilities, better quality output. Worse quality token probabilities, worse quality output. Watermarking changes the probability calculations for reasons other than quality. It can't not compromise quality. It literally leads the LLM to occasionally chose different tokens just for watermarking purposes.

It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator.

We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.

Fundamentally the article is railing against --temp > 0.0. He doesn't know what he's talking about.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#265

Post by vrganj »

wuschel wrote: post #204
epihelix wrote: post #50

Earlier quoted context omitted.

You know, back in the era when proofreaders were human, I never one met a proofreader who rewrote my text afresh, rather than annotating the text with a pen. It's still possible to use Claude to proofread - highlight grammatical, flow, structure, logic errors and make simple suggestions for you to pick and choose or adapt as you wish. No watermarking will flag your text. No flaw accusations of LLM authorship will hau…

Where is the problem with using LLM generated text? You could use your own hypothetical house elf to do it for you, or pay someone to do it. LLMs are just cheaper for a certain set of problems. People will find ways to circumvent this, so this limitation will only hit the technically less adept people.

Nobody says it's a problem. We'd just like to know.

Factory farming also makes meat cheaper than organic practices. I'd just like to know which one I'm getting.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#266

Post by Cakez0r »

thinkingemote wrote: post #247

I think we will see a hidden motivation behind this as ultimately so the output can have an author, the author can be attributed and finally the output will be copyrighted and so the LLM has more value. Everyone who invests in AI companies wants to see the value of their investment increase. I'd give it about 3-5 years until an AI company claims copyright over code their LLM produces. This is a crucial step in that p…

I think you're right that it's actually to do with attribution (i.e. They're not just watermarking your output as claude generated, but watermarking it as claude generated _by claude user id 73684_). I think it's more to do with the growing militarization of the internet. Just another brick in the wall of enter your phone number to create an account, send your id to prove your age, smile for the flock cameras, etc. Allowing the plebs to have privacy and anonymity is not allowed anymore.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#267

Post by Twey »

People are very upset, especially in the arts, that Anthropic is changing the text to watermark it, but isn't that missing the point a little bit? They're not changing _your_ text whose every word you've carefully chosen for the exact effect, they're changing text that they're generating, i.e. text you've already chosen to give up control over. LLMs can't understand emotional nuance anyway.

The phrasing of the announcement implying that phrasing and diction don't change the meaning of text is insultingly dismissive of the whole field of literature, and I can see why people might take it as an afront, but the actual technology shouldn't have a negative impact as far as I can see. It seems to me that this one is more of a PR problem than something with real-world impact.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#268

Post by josephg »

ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated.

I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#270

Post by carlosrg »

Gruber shows here that he really doesn’t understand the basics of how LLM text generation works. It’s weird he picked this battle about the quality of writing in LLMs. Was he planning to use LLMs to write his articles?

Well, not that weird actually. He just has a hard-on against anything that comes from the EU since Apple got in trouble. If the EU said tomorrow that they want peace in the world he’d be in Fox News the next day calling for an invasion. As a former reader of Daring Fireball, it’s just sad to see.

Post reply on HN