Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
1–10 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#2I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thing like write a story about Alice and Bob with each word starting in rotation with the letters alicebob, does it try to do so and hope there are roughly équiprobable tokens regularly?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#3> The provider must mandate in their terms-of-service that users not remove the watermarking.
So, you don't own the generated text, and can't use it freely then. What if I copy paste a section, or rewrite a section of text to my liking? What if I rewrite some lines of code that contains the mark?
Security theater, and vague enough to be used as a weapon against who the government wishes.
I hope it's left off for non-EU customers.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#4> the only acceptable answer for why an LLM should choose bananas instead of pineapple (or coconut, or guava, or papaya...) is that it has determined that it’s the best fit for the intended meaning, tone, and sentiment of the text.
It already fails. It randomly picks between close candidates. To help fool people into believing in intelligence claim, I guess.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#5I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thin…
> In any code or structured output, there just isn’t the flexibility
Variable name perversion incoming...
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#6I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thin…
Yes, and it says that in https://www.anthropic.com/news/claude-text-watermark
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#7Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude.
I think that was intended, yes.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#8I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thin…
Yes, and it says that in https://www.anthropic.com/news/claude-text-watermark
I should have read that, it’s actually quite reasonable and I don’t really understand the objections in TFA having read it.
> One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter.
Doesn’t make sense at all in light of the actual approach, they’re just choosing a different RNG. It’s not like they’re corrupting it by flipping words.
Should add I don’t support the watermarking and requiring it is idiotic.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#9> I want any LLM I use to choose the very best, most precise words at every single decision point.
Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo random. The output it generates is one of the possible outputs it would have generated before, just now it's deterministic and will generate the same thing every time.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#10Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.
It is quite just, if you think about it. Human works are copyrighted and protected at the moment of creation. All rights reserved. Yet, LLM outputs are uncopyrightable. Therefore, if Claude or any AI has processed my copyrighted work, the end result is uncopyrightable and in the Public Domain. The public has a right to know: is this a human copyrighted work, an LLM PD work, or is the human falsely claiming authorship in order to retain copyright?
A point of confusion for me, however: is every watermark unique? Is every algorithm for watermarking going to vary amongst models and amongst model versions? Will each model publisher keep this watermarking as a trade secret, that they alone can detect? If so, this can't scale! How do you detect "JoeBob 4.3 LLM" output? By querying every single model's watermark-detector? And if they all work by re-running the model and using tokens anew? That is extraordinarily wasteful.
If a watermark is not self-evident, or universally detectable, then it is no good. Take, for example, US currency. The security measures are published and well known. Any count-out room in retail has a big poster indicating how you can detect authentic US bills. Nobody has to accept non-US currency in the US, and so the only authenticity you need to worry about is your US bills alone. LLM watermarking has none of this in common. Currently sounding like a shitshow, if you ask me.