Claude is already a perversion of writing.
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
591–600 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#592My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
And I'm sure students will use tools to have every other paragraph written in the style of a different AI, in an attempt to defeat this fingerprinting.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#593Watermarks are context poisoning.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#594> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
His complaint seems to center on conversations between him and the LLM, not copy being written for publication elsewhere. That is, if Claude is going to teach him a new skill, he wants it to pick the most accurate words possible, not "pretty accurate words, subject to watermarking techniques".
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#595My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
And I'm sure students will use tools to have every other paragraph written in the style of a different AI, in an attempt to defeat this fingerprinting.
Yeah, wait for LLM "scrambles" that put every paragraph and then the whole text through multiple re-write/edit style cycles.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#596Earlier quoted context omitted.
Very intelligent people need intelligent-others to bounce ideas off of, and the LLM can be that.
Very intelligent people should be able to grasp that a LLM is not intelligent.
Have you tried to "rubber duck" an idea with an LLM. The latest models are pretty damnably good at it.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#597> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
He's a professional writer. As the article says, even if you write your own words, this is still a problem with proofreading, copy-pasting references or quotes, and "AI checkers".
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#598Earlier quoted context omitted.
> I don't think it's weird to ask that people commenting on x doing y with tech z at least try tech z, no? On what specific basis do you assume he hasn't tried it? He's definitely blogged about the desktop apps, after all. Or are you arguing that a writer doesn't have a meaningful or valid opinion on LLM-generated writing until they have tried to pass some off as their own? This just seems weird to me. I mean, I have…
I'm confused. You said "he doesn't use AI" and I took that as a general "he never used AI". If I was mistaken then ignore this whole thread, that's my bad.
Ahh — that was in the context of a suggestion AI-slop-writing I was replying to (quite an accusation for an established blogger IMO).
But one of the issues with HN threads is that you can sometimes lose the sense of what you're replying to by clicking further down the thread, and I have committed worse misunderstandings than this, so absolutely no need to apologise (and I probably need to consider this when I am replying) :-)
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#599Earlier quoted context omitted.
There are no watermark related tokens, there is no watermarked set - read the paper, it's public and not that complicated. It doesn't change the distribution of completions, and won't change the situation where one output token has the majority of probability mass. You're missing the fundamentals here!
I am going off the explanation in the declaude page (and related papers). But I see now anthropic mentions Aaronson's distortion-free watermarking. Random watermarking functions colour the tokens based on (small) contexts and a secret key. Given watermarking functions are randomly chosen every single time they are used (so essentially not deterministically seeded by the context and secret key), then indeed the comple…
For a given short-context (hash function seeding) you do have detectable manipulation of the completion (how they can read the marking). But, because (assuming enough entropy in context) the hashing function is decoupled from the log-probs, the perturbations are independent from the underlying distribution, so you're still sampling from the same distribution quotient some noise.
The only way you'd notice this is they weren't independent, and the most plausible way that happens is if you're re-completing pre-fills (resampling the same hash function against the same log-probs).
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#600Posting for visibility a reminder that this type of watermarking is comically easy to beat. See Scott Aaronson's 2023 research in which he coined the term "pineapple attack": one must simply prompt the model to add the word "pineapple" after every other word and then do a search + replace to remove the word, to obliterate the watermark altogether.
https://www.youtube.com/live/2Kx9jbSMZqA?si=0QgCPBX2_KPZ0QTU...