Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

131–140 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#131

Post by pizzly »

Art9681 wrote: post #73
levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

If this is true then the probability of the detection tools flagging completely human generated text as AI generated is non-trivial. Let's say I write a completely original piece and the detection tool says there is a 36% probability it was generated with Claude. What then? Now it's up to the person looking at the score to cast a subjective judgement. Maybe to me, anything over 25% is unacceptable. Maybe to someone e…

Worse, what will academic institutions decide is the threshold for detecting AI generated work. If you have a false positive how do you prove it was a false positive or we all just trust the watermark detector over the student saying "I swear I did it all by my self"

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#132

Post by inigyou »

amanzi wrote: post #88

I was initially surprised that Gruber was so invested in the "quality" of AI-generated text, which in my mind is an oxymoron. But really, Gruber's interest here is with the EU. This forms part of his ongoing attacks on the EU, all because they have been forcing Apple to align with regulations.

Can we install random unapproved apps on our iPhones yet, or is Apple aiming to just be fined a trillion dollars because they make more than that from the 30% cut?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#133

Post by inigyou »

jacobgold wrote: post #63

Watermarking will be one more nail in the coffin of proprietary models if the world is so fortunate. Reminds me of printer tracking dots. https://en.wikipedia.org/wiki/Printer_tracking_dots

And yet we still use printers and 90% of our color documents have the tracking dots.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#134

Post by beering »

cubefox wrote: post #99
beering wrote: post #70

Earlier quoted context omitted.

The watermark doesn’t change the distribution, only per-token selection. I think not understanding that is the source of most people’s FUD.

This comment disagrees with you: https://news.ycombinator.com/item?id=49324387

That comment merely says quality must be compromised. It doesn’t make it clear why that must be true. Empirical study seems to say that quality is not compromised, and looking at various proposed schemes, it seems intuitively true.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#136

Post by dofm »

codedokode wrote: post #57

Watermarks are garbage because they may embed account id, IP address and deanonimize you. That's why we should be using open-weights LLM whenever possible.

I don’t disagree about open weights (though the enabling aspect there is actually open source inference, right?)

But it feels to me like you would need a hell of a lot of text to bury even a simple account ID. The nudges they are talking about are of the order of a handful of bits over several hundred words, I think?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#137

Post by inigyou »

cubefox wrote: post #104
pibaker wrote: post #58

I think it's pretty dishonest of Anthropic to frame their watermark as EU regulation compliance. The EU regulation, from my understanding, requires AI content to be labeled for human viewers. In the meanwhile the Anthropic new release on the watermark says this. > The difference between watermarked and un-watermarked text will not be distinguishable to readers https://www.anthropic.com/news/claude-text-watermark Whic…

> The EU regulation, from my understanding, requires AI content to be labeled for human viewers. How would that work? Claude appending " written by AI" to each of its messages? That would both be impractical and useless.

I think there are two separate requirements? One that if you post something like an AI video on the internet or anywhere else, you must label it as AI. And another one that AI providers must watermark their outputs.

If you get caught uploading watermarked media without the clear label, you're in big trouble, mister.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#138

Post by veidr »

This (Anthropic's "watermark" stance, I mean) is so fundamentally ludicrous that I have assumed it is a (wholly insincere, but arguably pragmatic, at least from their perspective) attempt to deal with the EU and their latest misguided, ham-fisted attempt to solve a real-world problem by drenching the entire world with more regulatory slop[1].

The "watermark" can be trivially defeated, but may be enough to satisfy the letter of the law, and like many people here, I would argue that if you are letting Claude write for you, you've already accepted getting the literary equivalent of turd soup, so the harm is — or at least could be — fairly minuscule.

[1]: https://digital-strategy.ec.europa.eu/en/policies/code-pract...

(FWIW I have a more favorable view than most people seem to of the EU's efforts to at least try tackle problems like this — but predictably, the bureaucratic "solutions" they come up with don't work, but do make things objectively worse)

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#139

Post by addandsubtract »

alienbaby wrote: post #81
codedokode wrote: post #57

Watermarks are garbage because they may embed account id, IP address and deanonimize you. That's why we should be using open-weights LLM whenever possible.

This is the first post I've seen mention it. How traceable are the embedded codes?

There was an earlier instance of this here: https://news.ycombinator.com/item?id=48734373

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#140

Post by bagacrap »

> But only Anthropic will be able to determine if text was seemingly generated by Claude, and Anthropic will only be able to detect the watermarks that are applied by Claude. Claude can’t detect the hidden watermark signals generated by, say, Gemini, and Gemini can’t detect the hidden watermark signals created by Claude, because each implementation is predicated on secret keys held only by the LLM provider

Well, akshwally...

> Interoperability. Providers must implement an interoperability solution for watermark detection such as a standardized API access method, a publicly readable signpost mechanism embedded in content, or participation in a consortium detection solution by February 2, 2027

Post reply on HN