Are there some narrow cases (like in outputting the input almost verbatim) where the probability for every best token is always much higher than second best, and in those cases would there be no watermark?
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
201–210 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#202> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…
> [...] and go back and restructure sentences and paragraphs if we want.
That's what LLMs in reasoning mode do, too, to the text they present to you.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#203Earlier quoted context omitted.
If this is true then the probability of the detection tools flagging completely human generated text as AI generated is non-trivial. Let's say I write a completely original piece and the detection tool says there is a 36% probability it was generated with Claude. What then? Now it's up to the person looking at the score to cast a subjective judgement. Maybe to me, anything over 25% is unacceptable. Maybe to someone e…
> the probability of the detection tools flagging completely human generated text as AI generated is non-trivial How does that follow? AI-generated text is already not a perfect emulation of human writing. There's lots of room to affect it laterally without changing the level of quality. As I understand it, LLMs with temperature >0 can select from many possible outputs. All they're doing is limiting the possible outp…
To sample from the probability distribution you already need random numbers.
If you get your random numbers from a cryptographic PRNG, then to notice the difference between that and 'real' random numbers even in theory, means you need to break the cryptography. In practice, your gut feeling about how good some text is won't break modern cryptography.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#204Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.
You know, back in the era when proofreaders were human, I never one met a proofreader who rewrote my text afresh, rather than annotating the text with a pen. It's still possible to use Claude to proofread - highlight grammatical, flow, structure, logic errors and make simple suggestions for you to pick and choose or adapt as you wish. No watermarking will flag your text. No flaw accusations of LLM authorship will hau…
Where is the problem with using LLM generated text?
You could use your own hypothetical house elf to do it for you, or pay someone to do it. LLMs are just cheaper for a certain set of problems.
People will find ways to circumvent this, so this limitation will only hit the technically less adept people.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#205Earlier quoted context omitted.
> The EU regulation, from my understanding, requires AI content to be labeled for human viewers. How would that work? Claude appending " written by AI" to each of its messages? That would both be impractical and useless.
I think there are two separate requirements? One that if you post something like an AI video on the internet or anywhere else, you must label it as AI. And another one that AI providers must watermark their outputs. If you get caught uploading watermarked media without the clear label, you're in big trouble, mister.
We were talking about text.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#206> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
LLMs are no more than pen and paper at this point. Especially for those who aren't trying to create slop. We all would want our pens to accurately reflect the strokes (well in this case thoughts) rather than adding tiny watermarks to identify that it is generated by a particular pen or a user.
Watermarking per model is just the start. The method is cheap enough to distinguish individual users.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#207Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.
The question is, when is the "pro writer" version coming that lets you control this behaviour but costs more? Like night follows day, this will happen. They will need to dodge around the EU requirements but it will probably just come down to an alternative method to watermark or a contractual assurance you won't mis-represent the source of the text.
I had the same thought.
I hope the other providers will add a geographical limitation on this EU rule.
(On a side note, I wish they would replace those EU beauracts with LLms).
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#208Earlier quoted context omitted.
It’s fully possible I didn’t explain it very well in the first place, but he is making a wider point. The point I made (quite briefly) is that watermarking is only feasible because for good writing it is necessary to use T>0, or the writing will never explore a more creative choice, and that at T=0 you don’t even need a watermark to spot LLM-generated text. The point he is making is consistent with this, isn’t it? Ei…
The point he is making is not consistent with understanding how temperature influences LLM text generation, no. He repeatedly states that choosing "the best word" is the most important thing to him. I don't know how you reconcile that with creativity itself, let alone probabilistic sampling.
Yes, naively understood in his sense 'best' word means that you pick the word with the maximum score, instead of sampling from the distribution.
That doesn't actually give you the 'best' text in any human sense of the word. Just like playing the 'best' move in Poker without sampling leads you to lose a lot of money.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#209> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
LLMs are no more than pen and paper at this point. Especially for those who aren't trying to create slop. We all would want our pens to accurately reflect the strokes (well in this case thoughts) rather than adding tiny watermarks to identify that it is generated by a particular pen or a user. Watermarking per model is just the start. The method is cheap enough to distinguish individual users.
LLMs do a whole lot more than writing your thoughts down. They write extra text. If you just want a pen and paper, use Notepad. Or better, a pen and paper
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#210> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
One could also use butterflies to write ;)
https://xkcd.com/378/
The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work.
I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes."
I'm was using Claude Opus and now Fabel extensively for editing my texts and I find the recent updates abysmal. Not sure if it's due to the Text Watermark.
Before Claude was great in sharpening the meaning in my writing, it's now close to unusable.