Great question buried at the bottom:
> Also, what happens if another major global market makes it unlawful for AI to secretly watermark generated text?
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
481–490 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#482Earlier quoted context omitted.
> But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. This. I want the model I'm paying for to be "pure". I don't Anthropic or anyone else messing around with it, especially not for idiotic reasons like facillitating AI stigmatization. The "safety" nonsense is obnoxious enough. They should train the best possible model and let the weigh…
> especially not for idiotic reasons like facillitating AI stigmatization. From the people I’ve talked to at universities, LLM based cheating in education is an unstoppable nightmare. I don’t have a problem with LLMs. But I do want the cheating to - somehow - stop. The people who cheat miss out on learning. And the people who don’t cheat have their degrees devalued by those who do cheat.
> The people who cheat miss out on learning.
They aren't there to learn. They are there to jump through hoops to get a degree that will let them get a job so they can make money and prosper. The learning is entirely secondary.
The cheating will stop when there is no longer any economic incentive to be there in the first place. People with "pure" motives will refuse to cheat on their own, precisely because they want to learn and cheating prevents them from learning.
Watermarking AI output is a treatment for symptoms. The cause is the higher education meme. Somehow, getting a degree just became the default. Can't get a good job without a degree. That meme needs to die, and higher education will never recover its integrity until it does.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#483Earlier quoted context omitted.
> Why? I really don't understand this. Why can't a tech writer take a deep but neutral intellectual interest in something? Isn't it important that some do? Do people have to be stakeholders or clearly on one given team, pro- or anti-, for their opinion to matter? Is it that tribal? I don't think it's weird to ask that people commenting on x doing y with tech z at least try tech z, no? Imagine this pamphlet: Basting a…
> I don't think it's weird to ask that people commenting on x doing y with tech z at least try tech z, no? On what specific basis do you assume he hasn't tried it? He's definitely blogged about the desktop apps, after all. Or are you arguing that a writer doesn't have a meaningful or valid opinion on LLM-generated writing until they have tried to pass some off as their own? This just seems weird to me. I mean, I have…
I'm confused. You said "he doesn't use AI" and I took that as a general "he never used AI". If I was mistaken then ignore this whole thread, that's my bad.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#484Earlier quoted context omitted.
How exactly do you propose they should keep track of quality, then, if not by A/B testing?
The question here is not to what product management advice to the Gemini team. The discussion here is whether the watermarking is noticeable. The easy thing to do here would be to have 1000 questions, randomly assigning one half to an LLM with a watermark, and the other half without. Then show people pairs and say, "Which one seems watermarked?" (Or, "Which text seems more natural" or "Which is a better answer" or so…
They do exactly that "which is the better answer?" test -- I've seen it pop up a few times.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#485> “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter. I want any LLM I use to choose the very best, most precise words at every single decision point. Neither of these is "better" or "more precise"; in fact, LLMs will generally choose randomly between these candidates ba…
I’m more concerned that this will negatively impact code generation. An additional constraint completely unrelated to code quality is unacceptable as far as I’m concerned. I was an Anthropic user but now I’m looking at OpenAI or even better, open models.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#486My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
Doesn’t this mean Anthropic can accuse anyone of using their AI to write for them?
[dead]
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#487Earlier quoted context omitted.
You're generating a pseudo random number one way instead of another way. How would that inherently compromise quality?
Same reason that you don't just replace your rand() implementation with "return 4; // chosen by fair dice roll". If you need randomness for whatever reason, biasing the generator is compromising quality. ( https://xkcd.com/221/ ) In case of LLMs, you can look at it from high and low level. At low level - if you could do with less randomness, you can always lower temperature. You usually keep it (or for SOTA providers…
You're missing the same point that the blog post is missing. What they're doing is much less like replacing rand() with 4 and much more like setting seed(4) before generating any numbers. There is no "loss of quality" unless you're already using a temperature of 0.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#488Earlier quoted context omitted.
In practice, it is theater. Are they going to do this with the code output too? This is nonsense security theater for the low thinkers to have a sense that someone is in charge. When we all know nobody is in charge, anywhere.
Their AI model tends to write a lot of lengthy comment blocks. That's a fine place to put the watermark to track those users who accept the code blindly and don't delete/edit the comments.
The model doesn't particularly "know" which tokens within its output are "comments", it's not so easy
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#489My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
That's why detectors like Pangram exist too I think.
Pangram doesn't work.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#490> “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter. I want any LLM I use to choose the very best, most precise words at every single decision point. Neither of these is "better" or "more precise"; in fact, LLMs will generally choose randomly between these candidates ba…
I’m more concerned that this will negatively impact code generation. An additional constraint completely unrelated to code quality is unacceptable as far as I’m concerned. I was an Anthropic user but now I’m looking at OpenAI or even better, open models.
If you're concerned about code generation, then learn how to actually write the damn code yourself!