Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

321–330 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#321

Post by Closi »

TheOtherHobbes wrote: post #227
beering wrote: post #128

Earlier quoted context omitted.

> which inherently compromises quality. I don’t see how this follows? Tokens are chosen randomly. If you choose tokens with a different RNG in the same distribution, you’re still getting equally good or bad tokens.

Not all values of "equally good" are equally good. Writing has rhythm, or at least it's supposed to, and synonym swapping compromises it. Never mind metaphors and similes, which are even more tightly constrained. LLM writing is still a long way from good. Sometimes you get lucky with the odd line, but there's a difference in quality between influencer slop, genre fiction, and literary fiction and/or best-in-class jou…

> Not all values of "equally good" are equally good. Writing has rhythm, or at least it's supposed to, and synonym swapping compromises it.

We already know that a non-zero temperature improves quality though with current models (particularly with creative writing). The assumption that always picking the 'best' token results in the 'best' output is not the current reality.

And if you are already intentionally putting in randomness, I can imagine that it would be possible to seed the randomness in a way that is detectable but results in the same quality.

This is obviously not true for queries where temp = 0, but at temp = 0 then it becomes easier to identify anyway. I assume this technique implies some level of temperature.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#322

Post by nicbou »

LIMEVINCE wrote: post #258
nicbou wrote: post #219

Earlier quoted context omitted.

This is an excellent use case. It made learning German much easier. I write what I think is correct, then get a fixed version. When writing in English though, I use it more like a dictionary. If you want to write past a certain level, an LLM works better as a metaphor and idiom search engine. I also like to ask it to generate 20 ways to say the same thing. It’s a great way to simplify or smoothen sentences without lo…

Interesting, are you being literal about requesting 20 ways of saying the same thing? It seems pretty excessive and I'm actually impressed that asking an LLM to rewrite a statement 20 times yields output that isn't excessively redundant. Does the LLM do a pretty good job reading your mind, or do you still find yourself manually piecing together pieces from the 20 suggestions into a satisfactory sentence?

Yep, I literally ask it to say it in 20 different ways. Sometimes a sentence structure or a combination of words will just work better. The goal is usually to simplify a sentence without losing meaning.

I don't expect the LLM to read my mind. The unit of work is too small for intent to matter, and I'll just steer the next recommendations in a direction as needed.

Most of the suggestions are crap, but they can contain the seeds of a good sentence.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#323

Post by cookiengineer »

passwordoops wrote: post #314
chii wrote: post #274

Earlier quoted context omitted.

> blindly trusting they won't train on any of that being allowed to train on any data that you can legally obtain ought to be a right for anyone. After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.

You are one person. The corporation is not. Scale matters

> You are one person. The corporation is not. Scale matters

Correct, if you violate it too often to count, you have to pay around less than ~2.5ct per violation.

So the lesson here is: Create a company to do torrenting professionally, and resell its values for higher prices. Then get sued and pay a dime on the dollar you made.

edit: Actually it's 2.5ct per violation.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#324

Post by Alpha3031 »

ivan_gammel wrote: post #257
zahlman wrote: post #229

Earlier quoted context omitted.

> Where is the problem with using LLM generated text? In the fact that you didn't write it. > You could use your own hypothetical house elf to do it for you, or pay someone to do it. Yes, and those would be similarly problematic (and more expensive).

> In the fact that you didn't write it. This is a fact and this is generally not a problem. Customer support guy Joe did not write that email to you with a refund: someone else did it and Joe did pick the template. Alice did not write that post card to Bob, someone else did and she just googled some nice text. We deal with a lot of content that wasn’t written by the person who signed it. That content, when written by…

If it's not a problem, then there should be no issue with not concealing the fact, no? Lying about things one considers inconsequential is a useful signal about one's willingness to lie with little benefit.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#325

Post by simonh »

regularfry wrote: post #264
simonh wrote: post #239

Earlier quoted context omitted.

The quality of the LLM just _is_ the quality of the token probabilities it generates. Better quality token probabilities, better quality output. Worse quality token probabilities, worse quality output. Watermarking changes the probability calculations for reasons other than quality. It can't not compromise quality. It literally leads the LLM to occasionally chose different tokens just for watermarking purposes.

It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator. We already use an RNG at inference precisely because it leads to higher-quality output . Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user. Fundamentally the article is railing against --tem…

>It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator.

You can't as a user tell by how much the quality of the output was degraded. True.

>We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.

I'm not saying it wasn't random and now it is. I know how these things work. I said that the quality of the system is in the quality of the probabilities. That quality is being degraded.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#326

Post by [deleted] »

Arodex wrote: post #211
allisdust wrote: post #206

Earlier quoted context omitted.

LLMs are no more than pen and paper at this point. Especially for those who aren't trying to create slop. We all would want our pens to accurately reflect the strokes (well in this case thoughts) rather than adding tiny watermarks to identify that it is generated by a particular pen or a user. Watermarking per model is just the start. The method is cheap enough to distinguish individual users.

>LLMs are no more than pen and paper at this point. Then use pen and paper. It is the same, you say, right?

[deleted]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#327

Post by nicbou »

camillomiller wrote: post #255
nicbou wrote: post #219

Earlier quoted context omitted.

This is an excellent use case. It made learning German much easier. I write what I think is correct, then get a fixed version. When writing in English though, I use it more like a dictionary. If you want to write past a certain level, an LLM works better as a metaphor and idiom search engine. I also like to ask it to generate 20 ways to say the same thing. It’s a great way to simplify or smoothen sentences without lo…

As a writer, Claude’s metaphors are trite and obvious 90% of the time. It also has a terrible penchant for an immediately recognizable emphatic voice that makes even the best outputs super cringe. But people do not notice and do not care. In our bubble and John Gruber’s bubble we really overestimate how much people must care. Truth is, we’re in this predicament because statistically speaking the people who hive a fuc…

They really are. This is why I prefer the volume approach. I might not accept any of the ideas it spits out, but it often guides me in a direction I was not considering.

I know that most people don't care, but my online presence is a search query for interesting people, so I care about what I put into it.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#328

Post by gwd »

iainmerrick wrote: post #253
gwd wrote: post #243

Earlier quoted context omitted.

It says they observed no difference in people clicking thumbs up or down. There are loads of other behavior that they didn't observe; like, say, switching to a different LLM.

How exactly do you propose they should keep track of quality, then, if not by A/B testing?

The question here is not to what product management advice to the Gemini team. The discussion here is whether the watermarking is noticeable.

The easy thing to do here would be to have 1000 questions, randomly assigning one half to an LLM with a watermark, and the other half without. Then show people pairs and say, "Which one seems watermarked?" (Or, "Which text seems more natural" or "Which is a better answer" or something like that.) If they come out equal, the watermark really is indiscernible, at least to most people.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#329

Post by ppeetteerr »

Gruber was a good voice in the industry but this article misses the mark in a lot of ways.

A company the size of Anthropic would not voluntarily jeopardize their massive valuation if they didn’t feel the resulting output would maintain a similar level of quality as before. Is there a similar worry that their system prompt, which is injected at the start of every conversation also influences token generation in an artificial way?

If regulation will ruin Claude as a product, market forces will fill the void. There are also a ton of open weight models to choose from. It’s going to be okay.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#330

Post by frm88 »

vrganj wrote: post #261
TheOtherHobbes wrote: post #238

The issue here isn't (just) adulteration, it's that watermarking in general is unworkable. If all the providers use watermarking systems with different shifting logit weightings, and the keys are secret, you have to check every provider to see if it produced a given text. Which is clearly ridiculous. And if all providers collaborate and use the same weightings, or if the weightings are constant and not rotated crypto…

> Interoperability. Providers must implement an interoperability solution for watermark detection such as a standardized API access method, a publicly readable signpost mechanism embedded in content, or participation in a consortium detection solution by February 2, 2027

No idea why this was flagged to death. I vouched for it because it is a direct quote from the AI transparency act: https://theaicounsel.net/wp-content/uploads/2026/07/07_26_co... It clarifies interoperability requirements.
Post reply on HN