Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

381–390 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#381

Post by porridgeraisin »

KaiserPro wrote: post #278
boredhedgehog wrote: post #214

Earlier quoted context omitted.

> It seems to me like he started out mad and looked to justify it. Yes, but that's neither surprising nor a reason to dismiss the anger. People get angry about DRM schemes in video games, even if the slowdown these cause is practically imperceptible. They're angry -- and Gruber acknowledges that factor too -- because a stranger manipulates what they regard as their own domain, without consent by or benefit to the own…

> People get angry about DRM schemes Yes but thats a thing that degrades something in a catastrophic way, as in I can use the thing one day, and not the next. A different randomisation system on something that is a text generator which is designed to be unperceptable sounds like the people who are annoyed at FLAC vs MP3[1] Done right you won't know the difference, done badly and you will. [1] ex audio engineer, try m…

> try me

Good luck explaining that one

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#382

Post by lluisantoni »

Text watermarking is another EU rule made without real world input. The Union is stuck on major economic crises (electricity prices for instance) because nobody can agree on anything. However, the bureaucracy forces tech into a privacy nightmare. Brussels cannot bring together its own members but it loves pretending it can govern the internet.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#383

Post by holoduke »

Whole watermark thing is just bullshit. What if I add a watermark text and another AI as well. How many watermarks and who is the real creator. It just doesn't make sense. I ll eat my shoes if this concept is still a thing in 6 months

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#384

Post by kgarten »

voidUpdate wrote: post #216
kgarten wrote: post #210

Earlier quoted context omitted.

One could also use butterflies to write ;) https://xkcd.com/378/ The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work. I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes." I'm was using Claude Opus and now Fabel extensively for e…

I'll be interested to see if people can actually pick out which text is watermarked and which isn't once they introduce it. It won't switch "bananas" to "airplanes". It'll switch "I really enjoy eating bananas" to "I love eating bananas" or similar

That’s my point of the post … I had the feeling that the English text editing skills of Claude went significantly down in August.

I was frustrated at first not knowing what they are doing.

After I read this post, (seeing they introduced it in August) I think it has to do with the watermarking.

Try it on a paragraph … the connections between sentences feel clunky now.

I will play with it more and see if that’s really the case (the watermarking making the text edits worse).

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#385

Post by nullbio »

ThePhysicist wrote: post #338
ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

I'm sure they could use it to fingerprint people, at the very least.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#386

Post by vintermann »

ddalex wrote: post #356
ThePhysicist wrote: post #338

Earlier quoted context omitted.

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

Probably they bias the RNG for selecting the next token. This can be done practically in a lot of ways, including during training. I suspect the signal will be significantly under the noise floor, so it's not detectable if you don't know exactly what to look for, but certainly you can submit more information then the textual contents.

But if the user's prompt is in the context, you don't know exactly what the RNG chooses between. I don't know what trick they use to get past that, but it seems impossible to get by it in the general case (i.e. if the prompt can be anything) and you'll probably quickly compromise quality if you try.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#387

Post by SadErn »

ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

[dead]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#388

Post by pavlov »

bsenftner wrote: post #374
unbelievr wrote: post #361

Earlier quoted context omitted.

The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…

In practice, it is theater. Are they going to do this with the code output too? This is nonsense security theater for the low thinkers to have a sense that someone is in charge. When we all know nobody is in charge, anywhere.

Their AI model tends to write a lot of lengthy comment blocks.

That's a fine place to put the watermark to track those users who accept the code blindly and don't delete/edit the comments.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#389

Post by einpoklum »

Are we now going to live in a world in which people bitch and moan about large private corporations' LLM text generation service and whether it's good or bad etc.? As though they're supposed to be benevolent and serve the public interest? They're not and they don't. Also, write your own damn text.

> "My error was believing Anthropic"

The error is rearranging part of one's life around Anthropic.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#390

Post by DoneWithAllThat »

I know (and understand why) a lot of people cheer the EU’s increasingly vast regulatory environment as being “pro consumer” but I’m really tired of said regulations being inflicted on the rest of the world. If this is what Europeans want for themselves that’s fine. But I no more want their regulations to be the de facto world’s any more than I want China’s.
Post reply on HN