Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

181–190 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#181

Post by [deleted] »

Maxatar wrote: post #119
brookst wrote: post #68

Earlier quoted context omitted.

This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.

>There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”. You're mixing up two claims here, and only one of these is kind of true. Yes LLMs do internally plan ahead in a way that is emergent rather than strictly part of their architecture, so that part of your claim is true. The way you word it by saying they are "coalescing the probabilities of a range of t…

[deleted]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#182

Post by nextaccountic »

Maxatar wrote: post #119
brookst wrote: post #68

Earlier quoted context omitted.

This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.

>There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”. You're mixing up two claims here, and only one of these is kind of true. Yes LLMs do internally plan ahead in a way that is emergent rather than strictly part of their architecture, so that part of your claim is true. The way you word it by saying they are "coalescing the probabilities of a range of t…

Reasoning tokens are a way to escape autoregressive woes. The model can generate a draft, then ponder on it, and use this to generate a final version

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#183

Post by cryptonector »

Gigachad wrote: post #66
Imnimo wrote: post #51

>I want any LLM I use to choose the very best, most precise words at every single decision point. Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? This entire article just seems so detached from the basics of how LLMs work.

I think the author is just mad people will be able to detect and filter out their AI slop writing in the future.

People already are, and do.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#184

Post by andOlga »

What a truly bizarre article. Arguments about pre-existing randomness, temperature and whatnot aside, I simply cannot comprehend what the author here really thinks the "best word" is. There's no such thing. We humans fall on familiar patterns of writing ourselves, so we may forego something with a flourish in favor of a more commonly-used word unless we put in effort to be "special", which should be used sparingly. That is to say, human writers are likely to choose a "worse" word in far more than the supposed 51% of cases, and that has no effect on the actual quality of writing in the end.

But even if there were such a thing as a truly "best word", for some context, what are the examples here? Mango vs pineapple? Gray vs overcast? In what case is one of these better, that AI would normally infer but would suddenly be "perverted" by SynthID? Do you think your emotional state and preferences are being evaluated if they aren't explicitly in memory? And if they are there, do you think that the generator will bypass those instructions in favor of the watermark instead of placing it somewhere you won't care? I just. Genuinely don't get it. There may be words that matter in specific contexts or to you as a reader, so you should bloody well put them there.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#186

Post by rsynnott »

inigyou wrote: post #132
amanzi wrote: post #88

I was initially surprised that Gruber was so invested in the "quality" of AI-generated text, which in my mind is an oxymoron. But really, Gruber's interest here is with the EU. This forms part of his ongoing attacks on the EU, all because they have been forcing Apple to align with regulations.

Can we install random unapproved apps on our iPhones yet, or is Apple aiming to just be fined a trillion dollars because they make more than that from the 30% cut?

Yes: https://support.apple.com/en-mk/117767

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#187

Post by avazhi »

breezybottom wrote: post #159
avazhi wrote: post #103

[flagged]

Clearly Anthropic thinks its more profitable to comply and have access to the European market, but I'm sure you know better than the people who brought it to a $2 trillion valuation.

I didn't say anything about profitability. Since Anthropic is all about the good of mankind etc, ostensibly profitability isn't their number one priority.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#188

Post by krackers »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

>fails to understand the gumbel softmax technique

I think in this case it doesn't help that there are multiple watermarking schemes, and the easiest for people to understand is the red/green scheme by Kirchenbauer et al. (https://arxiv.org/pdf/2301.10226), which does technically distort the logits (but I'd argue only in cases where you wouldn't notice it anyway).

I wasn't aware of this gumbel softmax scheme, it seems you're referring to https://simons.berkeley.edu/talks/scott-aaronson-ut-austin-o... ? That's really clever as it doesn't even distort the logits, basically cryptographically indistinguishable from a "real" random sample unless you have the key.

The actual scheme Claude uses seems to be neither of those two though, they say it is SynthId-text which seems to be tournament sampling based.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#189

Post by dexterlagan »

I needed another reason to cancel my Claude sub. Thanks Anthropic!

This is akin to adding a giant watermark on things one would made with a free product "Made with XXX". Except you're paying $200/month for it, and there's no way to disable that watermark.

I don't disagree with EU regulations, but I strongly believe the onus should be on the content publisher, not the toolmaker. If the toolmaker watermarks whatever his tool produces, it opens a giant can of worms that cannot be closed. That means anything and everything you make with this tool is no longer fully yours, it contaminates everything and makes your work traceable. Who wants that?

I was already annoyed by the fact that Claude marked everything it did on my repos under its own account (I didn't ask for any of this), but now everything is invisibly marked, even the code. Not that I care that my writing would be watermarked, since I'd rather write my stuff myself, but code? No thanks.

Meanwhile I'm running a DeepSeek V4 Flash or Pro, or a Qwen3.8, and it writes my code without a peep. Resulting repos are clean, just the way I want them. No 'Claude' account, no watermarking, nothing. I won't be looking back after having tried these new models. Whoever makes good models that don't broadcast their maker will get my business.

This watermarking will simply push people more towards Chinese models. Keep pushing in the wrong direction Anthropic. Doing this right before an IPO is a great idea.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#190

Post by slhck »

The fact that Gruber points to a "must-read" article about how the watermarking works, which, in turn, was very obviously entirely LLM-generated, says a lot about his lack of experience reading LLM output.

If Gruber can't tell a fully AI-generated article from a human-written one, perhaps he shouldn't care so much.

NB: I was told yesterday it's apparently a meme to call out Claude-generated output, but here I am, as I believe it's quite relevant to the topic at hand.

Post reply on HN