What a strange take. LLMs themselves are a perversion of writing. I couldn't care less about the implementation details of the PRNG they use for next-token sampling (well, as long as they're not stuffing a user ID in there).
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
351–360 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#352Earlier quoted context omitted.
> being allowed to train on any data that you can legally obtain ought to be a right for anyone. I have the opposit viewpoint to the extreme. They shouldn't be allowed to even read that data until they are very clear about what they will or not do with it. Can they publish it? Can they store it? Can they use the information in it on prediction markets? Etc. Humans reading texts historically come with little negative…
> Humans reading texts historically come with little negative consequences, but machines reading and processing texts en masse is more dangerous and should be regulated. Citation needed. This is sounding tautological.
Not really hard to find example: License plate ALPR like Flock ?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#353Earlier quoted context omitted.
I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated. I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
> Any stenographic system that you have the code for can be trivially defeated. They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it. Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're…
I assume this oracle will be behind 20 layers of anti-bot protection, CAPTCHAs and hardware attestation challenged. It will be incredibly painful to use. It won't stop the motivated attackers, but will make it too annoying for the average person.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#354Earlier quoted context omitted.
I mean this is the thing that really comes off hard. If you want precision and clarity of your writing, then you need to hand write it. Just like when you are optimising, its common to drop to a lower level language because the compiler doesn't express what you want. Sure its hard, but you know, thats kinda the point. Even if you don't want that, the LLM is an average of the style it was trained to give out. Which is…
The bigger problem is its effect on the way we use language. The amount of LLM generated or edited media is going to keep increasing and the media that people consume affects their own word choice.
> affects their own word choice.
exactly. in the same way that printed books affected word choice, so did the radio.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#355So the watermark can be removed by rearranging words and choice of words. This seems trivial to bypass with a local model. If I understand this correctly.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#356My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…
Probably they bias the RNG for selecting the next token. This can be done practically in a lot of ways, including during training.
I suspect the signal will be significantly under the noise floor, so it's not detectable if you don't know exactly what to look for, but certainly you can submit more information then the textual contents.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#357> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
What an absurd take on a very reasonable concern. As much as I hate “obviously AI” writing, I don’t understand why we have to handicap the tech and prevent it from improving. This manipulation of word choices virtually guarantees AI will always sound like a robot.
Reasonable? A concern which is based on no real data?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#358Earlier quoted context omitted.
I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated. I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
> Any stenographic system that you have the code for can be trivially defeated. They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it. Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're…
They could (..and probably will..) store that version and then refuse the check if this attack is detected, i.e. the version is too close to a known LLM output.
Alternatively they could also just keep saying "yes" if it's close enough to a version that was close enough.. Although that would enable the attack to allow arbitrary text to be "proven" AI, by slowly morphing close-enough generated material to the desired text. But perhaps this is not a problem they are not concerned with.
To satisfy the letter of the law I expect it's enough to just provide the oracle, without any mitigations.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#359My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
> blindly trusting they won't train on any of that being allowed to train on any data that you can legally obtain ought to be a right for anyone. After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.
> After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read).
Are you a tool?
Because humans gets rights, tools don't.
Arguing that untrained or partially trained models should have have rights is a different argument to arguing that a trained model should get the same rights as a human.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#360Author read the posts but didn't understand them.
> At each decision point, they’re a little more likely to pick a word from the green list than the red list.
Wrong. There is no global red and green list. It's dependent on context and balances out on average. It won't change the result when one token is predicted overwhelmingly likely.