Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

271–280 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#271

Post by piker »

josephg wrote: post #268
ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated. I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!

Won't we just be able to fine tune OSS models to detect these patterns across providers? It will be cat-and-mouse but my bet is it converges to a central detector that isn't affiliated with any model provider.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#272

Post by soVeryTired »

voidUpdate wrote: post #216
kgarten wrote: post #210

Earlier quoted context omitted.

One could also use butterflies to write ;) https://xkcd.com/378/ The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work. I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes." I'm was using Claude Opus and now Fabel extensively for e…

I'll be interested to see if people can actually pick out which text is watermarked and which isn't once they introduce it. It won't switch "bananas" to "airplanes". It'll switch "I really enjoy eating bananas" to "I love eating bananas" or similar

To that point, given a corpus of writing from person A and another from person B, I have no doubt it’s easy to train a classifier to determine who wrote what. In fact I believe law enforcement agencies already have these classifiers.

The only difference here is that Anthropic is actively trying to make the watermark undetectable.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#273

Post by KaiserPro »

voidUpdate wrote: post #199

> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much

I mean this is the thing that really comes off hard.

If you want precision and clarity of your writing, then you need to hand write it. Just like when you are optimising, its common to drop to a lower level language because the compiler doesn't express what you want. Sure its hard, but you know, thats kinda the point.

Even if you don't want that, the LLM is an average of the style it was trained to give out. Which is a homogenisation of the language to create a vague padding medium between a few generalised facts. (because a. it makes it less jarring when stuff is wrong, because its smeared over a higher amount of text and b. it looks more 'professional' because American business English is all guff and no meat)

Also yes, two isolated phrases may have subtly different meaning, frankly, the nuance is missed on most people. If you look at the interactions on here, at least 25% of the arguments are caused by people angrily reacting to the things _they_ thought the other person was saying, rather than what the actual person was saying.

So no its not a perversion, the LLM is, if you're gonna be picky about things.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#274

Post by chii »

ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

> blindly trusting they won't train on any of that

being allowed to train on any data that you can legally obtain ought to be a right for anyone.

After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#275

Post by neuroticnews25 »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

>It does not affect writing quality at all, provably

Prove it, then? It's not a claim that GumbelSoft paper makes: "Regarding generation quality (perplexity), GumbelSoft shows relatively low perplexity"

https://arxiv.org/html/2402.12948v3

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#276

Post by Cakez0r »

andy99 wrote: post #2

I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thin…

This could be why claude code has recently started to write reams of inane comments alongside the code it generates.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#277

Post by knollimar »

inigyou wrote: post #137
cubefox wrote: post #104

Earlier quoted context omitted.

> The EU regulation, from my understanding, requires AI content to be labeled for human viewers. How would that work? Claude appending " written by AI" to each of its messages? That would both be impractical and useless.

I think there are two separate requirements? One that if you post something like an AI video on the internet or anywhere else, you must label it as AI. And another one that AI providers must watermark their outputs. If you get caught uploading watermarked media without the clear label, you're in big trouble, mister.

And it mandates that Anthropic put it in their TOS rather than make it law.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#278

Post by KaiserPro »

boredhedgehog wrote: post #214
wpietri wrote: post #78

Earlier quoted context omitted.

It seems to me like he started out mad and looked to justify it. I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists. If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experim…

> It seems to me like he started out mad and looked to justify it. Yes, but that's neither surprising nor a reason to dismiss the anger. People get angry about DRM schemes in video games, even if the slowdown these cause is practically imperceptible. They're angry -- and Gruber acknowledges that factor too -- because a stranger manipulates what they regard as their own domain, without consent by or benefit to the own…

> People get angry about DRM schemes

Yes but thats a thing that degrades something in a catastrophic way, as in I can use the thing one day, and not the next.

A different randomisation system on something that is a text generator which is designed to be unperceptable sounds like the people who are annoyed at FLAC vs MP3[1]

Done right you won't know the difference, done badly and you will.

[1] ex audio engineer, try me.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#280

Post by Gud »

chii wrote: post #274
ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

> blindly trusting they won't train on any of that being allowed to train on any data that you can legally obtain ought to be a right for anyone. After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.

Why would that be a legal right?

Why should we hand over even MORE power to the owner class?

In a fantasy world this could be possible yes.

Post reply on HN