If you don’t like it don’t use it
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
581–590 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#582A question not addressed here is whether there exists any solution to the EU requirement that doesn't suck.
If the same red/green algorithm described in the article is applied to generated code, I cannot imagine how that does not degrade code quality (probabilistically, not at every point).
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#583Earlier quoted context omitted.
Gruber just hates any kind of EU regulation of US tech companies ever since they started making what he calls "product decisions" for Apple.
As an EU citizen and user of Apple products, I feel the same
We're all entitled to our opinions, but then just say that, don't go to great lengths to misunderstand and justify technology that you don't even use yourself to justify why the regulation is bad, just say "I think regulation is fundamentally bad".
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#584Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).
It's a writer perspective versus a reader perspective maybe?
Sometimes when you're trying to write something, it really seems like the exact words matter a lot. Suggestions made to be more direct or use a more common word here or whatever seem to really impact the thought that you're trying to communicate.
Certainly we've all had times when trying to communicate clearly when the specific words seem very important.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#585Earlier quoted context omitted.
Whereas if you fully avoid the letter e, everyone will know you are George Perec
And if you lowercase everything, you're e e cummings.
most people have at least two, but generally around four to six (or more, thanks internet) interactional styles (not "selves", just things which vary depending on context, subject, and people they are interacting with). probably more. some of it is due to simple physical comfort levels (right now I am in a physical position where capitalizing is more difficult); some might be due to talking to a peer group instead of a group of kids or a priest or boss at a job, etc. unsure how that will shake out with AI but it unnerves me.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#586Earlier quoted context omitted.
Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…
The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…
[dead]
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#587> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…
Another good way to think about this is that it does change the output, but in a way that is equally likely to make it "better" as it is to make it "worse".
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#588I've been seeing a lot of the same questions about watermarking over the last week, so I made this playground that lets you try out 3 different watermarking schemes (including a version of SynthID-Text) on your own text: https://watermark.keito.me/ to understand it experientially myself. I found it illustrative to try different examples like code, text rewriting, etc. to see how it affected the tokens. Playing around with the detector is interesting too to see how much of the watermark can remain in edited text. I hope this is helpful for others too.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#589My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
> checking any text for watermarks requires sending the entire text to Anthropic Couldn’t it be checked in the TEE using confidential computing to keep Anthropic’s algorithm secret?
I can't speak for Anthropic, but with Google's SynthID, the algorithm is actually public. However, checking (or creating) the watermark requires a symmetric key, and the providers likely wouldn't share that key.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#590I’m surprised by the comments here being so favorable to anthropic. The comments are right about there being no “best” token, and yeah Gruber may have an agenda here. But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. Take the “gray” and “overcast” choices. And lets say before applying synthid the percentages were 48% and 52%. Those pe…
> To change those percentages to 45% and 55% This seems like a fundamental misunderstanding of how this sort of watermarking works. (Either that, or I have a fundamental misunderstanding of how it works lol.) It doesn't change the probability distribution of the next token at all. If you were getting XYZ 48% of the time before, you're still getting XYZ 48% of the time. What's changed is where the random numbers come…
huh, thanks for commenting this! I trusted the linked explanation post https://declaude.org/watermarking/ but actually reading the the synthid paper showed me that my understanding was wrong: https://www.nature.com/articles/s41586-024-08025-4
It is definitely blurrier whether you can say this approach changes the distribution then. By definition, it _has_ to change the probabilities of output tokens, but it's not totally clear that the pseudorandomly generated scoring functions does affect the learned distribution.
put another way, I think it's safer to do:
compute distribution -> sample -> watermark from sampled options
than it would be to do:
compute distribution -> watermark distribution -> sample