The author makes a lot of great points. I find it surprising that somebody who has such a nuanced appreciation for the subtleties of language would be in the crowd complaining about the watermarking policy. I expected this kind of complaints from mostly students interested in academic dishonesty, who generally don't have enough command over written language to notice the slight decrease in output quality.
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
251–260 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#252I often agree with John Gruber, but I think he’s lost the plot with this one. The thing I don’t understand is why he seems to care so damned much about this subject--enough to write over 4,500 words on it! John writes for a living. That’s his profession. He’s been writing for over 25 years now. When you’re that good at writing, and you care this much about your writing, you don’t allow an LLM to take over your job. I…
This guarantees he can never use it for the one purpose he might care about though
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#253Earlier quoted context omitted.
Google has A/B tested watermarking on millions of responses. They say they observed no difference in user behavior.
It says they observed no difference in people clicking thumbs up or down. There are loads of other behavior that they didn't observe; like, say, switching to a different LLM.
How exactly do you propose they should keep track of quality, then, if not by A/B testing?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#254Earlier quoted context omitted.
One could also use butterflies to write ;) https://xkcd.com/378/ The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work. I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes." I'm was using Claude Opus and now Fabel extensively for e…
I'll be interested to see if people can actually pick out which text is watermarked and which isn't once they introduce it. It won't switch "bananas" to "airplanes". It'll switch "I really enjoy eating bananas" to "I love eating bananas" or similar
[deleted]
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#255Earlier quoted context omitted.
One could also use butterflies to write ;) https://xkcd.com/378/ The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work. I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes." I'm was using Claude Opus and now Fabel extensively for e…
This is an excellent use case. It made learning German much easier. I write what I think is correct, then get a fixed version. When writing in English though, I use it more like a dictionary. If you want to write past a certain level, an LLM works better as a metaphor and idiom search engine. I also like to ask it to generate 20 ways to say the same thing. It’s a great way to simplify or smoothen sentences without lo…
As a writer, Claude’s metaphors are trite and obvious 90% of the time. It also has a terrible penchant for an immediately recognizable emphatic voice that makes even the best outputs super cringe.
But people do not notice and do not care. In our bubble and John Gruber’s bubble we really overestimate how much people must care. Truth is, we’re in this predicament because statistically speaking the people who hive a fuck are a rounding error.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#256LLM output, as the author acknowledges here, is already non-deterministic. Next token probabilities are set, and tokens are chosen pseudo-randomly. As I understand it, this watermark is just going to be a matter of using a known seed and algorithm to make those pseudo-random choices, such that a signature can be detected. The important thing is, it's not replacing intentional choices with random ones, it's just gener…
That cannot be true. The quality of an LLM's output is the quality of the probability calculations for the next token. Anything that degrades the relationship between the system's best assessment of the appropriate probability and the actual probability used is a degradation of the quality of that probability and therefore of the output. If this didn't have a detectable effect on the quality of the token probability…
An encrypted hard drive is EXACTLY uniformly distributed random bytes if you do not know the encryption key. No one would be able to tell the difference between a drive that is just purely random numbers or is actually filled with content. (Of course excluding the usually intentionally added readable header)
If this was not the case, the encryption would be broken, and most everyone agrees that good encryption does exist.
The "quality of the probability calculations" as you put it is 100% in this case and any less would be a huge deal (as in - breaks all of the internet).
So, now you just take those same random bytes and use them as the seed for your LLM token choices. The output has the _cryptographically_ proven exact same quality as if you were using a true RNG (which you likely weren't using anyways).
You just need to know your LLM distribution and the encryption key, then with each new token you exponentially increase the chance of knowing whether it fits your encryption key. Without actually affecting the token choice in a perceivable manner.
> choosing different words that it otherwise would
The "that is otherwise would" is carrying all the weight here. "Otherwise" is sampling from a distribution. You just sample from the same distribution but with a cryptographically secure, seeded RNG. https://en.wikipedia.org/wiki/Cryptographically_secure_pseud...
"Knowing the LLM distribution" seems to me like the only hard part because you don't know the context of any random snippet.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#257Earlier quoted context omitted.
Where is the problem with using LLM generated text? You could use your own hypothetical house elf to do it for you, or pay someone to do it. LLMs are just cheaper for a certain set of problems. People will find ways to circumvent this, so this limitation will only hit the technically less adept people.
> Where is the problem with using LLM generated text? In the fact that you didn't write it. > You could use your own hypothetical house elf to do it for you, or pay someone to do it. Yes, and those would be similarly problematic (and more expensive).
> In the fact that you didn't write it.
This is a fact and this is generally not a problem. Customer support guy Joe did not write that email to you with a refund: someone else did it and Joe did pick the template. Alice did not write that post card to Bob, someone else did and she just googled some nice text. We deal with a lot of content that wasn’t written by the person who signed it. That content, when written by LLM, may indeed contain watermarks and nobody will care about the choice of words, because only the meaning matters in such communications.
People pay too much attention to authenticity here, which is no more than a demonstration of an effort. LLM can and should write scientific articles because the real effort is in directing research, not summarizing it. LLMs can and should write news, because it is cheap and efficient, and real reporting is in discovery. LLM can and should write fiction and make movies, because there is no reason why creators of various junk should earn their money easily. LLMs do not replace real talent. They just emphasize for an average person how easily replaceable they are. And that‘s ok. Creative industry is a blue collar job now.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#258Earlier quoted context omitted.
One could also use butterflies to write ;) https://xkcd.com/378/ The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work. I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes." I'm was using Claude Opus and now Fabel extensively for e…
This is an excellent use case. It made learning German much easier. I write what I think is correct, then get a fixed version. When writing in English though, I use it more like a dictionary. If you want to write past a certain level, an LLM works better as a metaphor and idiom search engine. I also like to ask it to generate 20 ways to say the same thing. It’s a great way to simplify or smoothen sentences without lo…
Interesting, are you being literal about requesting 20 ways of saying the same thing? It seems pretty excessive and I'm actually impressed that asking an LLM to rewrite a statement 20 times yields output that isn't excessively redundant. Does the LLM do a pretty good job reading your mind, or do you still find yourself manually piecing together pieces from the 20 suggestions into a satisfactory sentence?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#259tl;dr:
> It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of (watermarking)
And an example of the impact of watermarking on word choice [0]:
> The results of the study were quite [important || significant || substantial || notable]
The meaning of the sentence to changes slightly even in just this tiny example. Imagine the degradation when applied across an entire response!
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#260Earlier quoted context omitted.
It seems to me like he started out mad and looked to justify it. I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists. If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experim…
> It seems to me like he started out mad and looked to justify it. Yes, but that's neither surprising nor a reason to dismiss the anger. People get angry about DRM schemes in video games, even if the slowdown these cause is practically imperceptible. They're angry -- and Gruber acknowledges that factor too -- because a stranger manipulates what they regard as their own domain, without consent by or benefit to the own…
Even if honor were real, LLMs do not have honor, and people using LLMs to write without disclosing that fact (or indeed at all, to some purists) do not have honor either.
> because a stranger manipulates what they regard as their own domain, without consent by or benefit to the owner.
It's LLM output! It's not your domain, it's the LLM owner's!