Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

701–710 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#701

Post by mrcslws »

mangoman wrote: post #659
Nimitz14 wrote: post #628

Earlier quoted context omitted.

You’re not getting it. The probabilities do not change at all. The only change is given some probabilities there is a deterministic method for determining which symbol was sampled from that distribution. The distribution or sampling process itself is not modified.

Based on the SynthID-Text paper https://www.nature.com/articles/s41586-024-08025-4 I agree that the LLM's learned distribution isn't modified, but I don't think it's correct to say that the sampling process is not modified. Also I just read the paper today so I could be misinterpreting things. As described in the paper, you're right that it doesn't affect the main sampling technique, but what they do is they sample t…

I think you're correct. It does alter the distribution for each output token, implicitly giving each candidate token a different probability. Maybe that's fine, but it's not as magical as Anthropic [and a lot of commenters here] are making it out to be.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#702

Post by Lumich »

einpoklum wrote: post #672
Lumich wrote: post #575

Earlier quoted context omitted.

Spot on. Also, in 2003, the war on Iraq, still occupied. And the proxy war on Syria (stifling an unwelcome pipeline project). European “leaders” pretend to not comprehend how they're being screwed. Stockholm syndrome. Populations don't understand, propaganda (“free press”) working correctly.

It's very weird looking in from the outside. I mean, sure, the US is the dominant power and everything, so things like Iraq and Iran could be construed as just collateral damage from their imperial maneuvers. But it has just piled on, more and more, and even when they are hit right in the face with the massive bombing of NordStream, still practically nobody tries to draw any sort of line. Reminds me a bit of that 'Ye…

Thanks for that YPM sketch. Incidentally, I think the “nuclear deterrant” story is total bogus from the get-go. But has worked great for decades as a framework to frighten people, or at least provide justifications and impose “global order”.

The trouble with drawing any sort of line is that when you're complicit and “aiding and abetting”, there's simply no motivation. And as long as the populace doesn't have a clue what's going on, their understanding being reduced to “evil Putin”, that's not even much of a problem.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#703

Post by psjs »

Y-bar wrote: post #543
KaiserPro wrote: post #474

Earlier quoted context omitted.

> Neither can I recall there has ever been a moveable type press, Ah my friend, you are about to fall down a rabbit hole into standardisation of spelling, and the sometimes deadly debates about how to translate latin into the vernacular. English, as she is written, is a great example. for the spoken word, BBC/received pronunciation is another. I speak the way I do _because_ of BBC radio. The reason I have the accent…

A radio device at BBC did all this and not humans? > again, your argument is against LLMs and globalisation of culture, not finger printing. ABSOLUTELY NOT. Do not ever put words in my mouth. My argument is that radio is a medium through which humans communicate. An LLM is not.

[1] > A radio device at BBC did all this and not humans?

Technology mediates (human) agency.

[2] > My argument is that radio is a medium through which humans communicate. An LLM is not.

KaiserPro's argument is that radio (a one-way medium) mediates how humans communicate (phonetically), thus influencing how people speak.

Now that does not explain how the printing press and radio have influenced word choice (both of which I would like to see examples of!)

[3] > I can't remember any radio determining words or adjusting grammar of the person speaking through it.

Now media themselves did not really have something that looked like the kind of pseudo-agency that LLMs seemingly have. There may be some kind of qualitative leap.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#704

Post by phiresky »

simonh wrote: post #331
phiresky wrote: post #256

Earlier quoted context omitted.

An encrypted hard drive is EXACTLY uniformly distributed random bytes if you do not know the encryption key. No one would be able to tell the difference between a drive that is just purely random numbers or is actually filled with content. (Of course excluding the usually intentionally added readable header) If this was not the case, the encryption would be broken, and most everyone agrees that good encryption does e…

>You just need to know your LLM distribution and the encryption key, then with each new token you exponentially increase the chance of knowing whether it fits your encryption key. Without actually affecting the token choice in a perceivable manner. I struggle to understand the relevance of that comment. The blue/green token list biasing process literally does cause different tokens to be occasionally chosen. Not only…

Well I was assuming your comment was saying it's _impossible_ to do this without affecting output, so I was just describing an option on how it IS possible. I saw that token list biasing post and yes that looks pretty bad. I didn't look in enough detail to see whether it's actually clear what Anthropic and SynthID really do. I would assume though their approach is of the smart kind.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#705

Post by llm_nerd »

Marsymars wrote: post #690
llm_nerd wrote: post #308

Earlier quoted context omitted.

Gruber has long been a talented, excellent writer. I hugely doubt he uses AI at all, nor does he plan to. His first take on this situation was cutely naive, thinking they were going to inject secret hidden unicode characters. But ultimately he has a massive hate on for the EU -- they were mean to Apple once -- and it comes out in any topic that overlaps.

> Gruber has long been a talented, excellent writer. I hugely doubt he uses AI at all, nor does he plan to. He notes in various other posts that he uses AI/LLMs and chatbots quite extensively. (I don't recall what for exactly, but not for writing his pieces.)

Good point, and I should have been clearer that I don't think he uses or will use it for writing. He's far too skilled of a writer to need it, and at best it would be a handicap.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#706

Post by StevenWaterman »

Marsymars wrote: post #695
baobabKoodaa wrote: post #691

Earlier quoted context omitted.

You will understand if you run some LLM models with a greedy sampler that does that. The text quality begins to deteriorate rapidly. This is a very counter intuitive result so I don't blame you for not understanding until you actually tried it and experienced it for yourself.

> You will understand if you run some LLM models with a greedy sampler that does that. The text quality begins to deteriorate rapidly. Right, I've done this, and this makes sense to me, but I'm not following how that falsifies the top probability word being the best choice in any particular instance. "Picking only the best word at each decision point results in a worse final result" seems like an imminently reasonabl…

I think you're using different definitions of best. If best = leads to a correct answer overall then by definition anything that leads to a bad outcome can't be best

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#707

Post by kgarten »

kergonath wrote: post #527
kgarten wrote: post #384

Earlier quoted context omitted.

That’s my point of the post … I had the feeling that the English text editing skills of Claude went significantly down in August. I was frustrated at first not knowing what they are doing. After I read this post, (seeing they introduced it in August) I think it has to do with the watermarking. Try it on a paragraph … the connections between sentences feel clunky now. I will play with it more and see if that’s really…

> After I read this post, (seeing they introduced it in August) I think it has to do with the watermarking. The power of confirmation bias… > Try it on a paragraph … the connections between sentences feel clunky now. We might have a definitive explanation at some point, but there are about a dozen possible reasons for something like this. For starters, is this something really significant and not something you notice…

I’m frustrated since August with the results from Claude for English text editing … I didn’t look for that. It’s just a fact.

This is not about em dashes …

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#708

Post by llm_nerd »

ragazzina wrote: post #630
llm_nerd wrote: post #303

Earlier quoted context omitted.

Gruber went from the naively wrong claim that it would insert secret hidden characters (which would be trivial to remove, obviously), to quickly writing a giant essay as if he's an expert on LLMs. Like you said, he is strangely fixated on the EU, and is certain any EU rule is the worst thing in the universe, and this whole piece seems motivated by that guiding force. Further he later compares Gemini to Anthropic mode…

How is he "strangely" fixated on EU? The EU tries to limit Apple's power, Gruber shills for Apple, Gruber is against the EU. Nothing strange about it.

I completely agree with you. But that doesn't change that Gruber's fixation on the EU is strange, specifically because he's so ego-tied to Apple that he strikes out like a untrained chihuahua at his list of Apple adversaries. He has done this for many years and it's just weird.

It's like some weird k-pop stan sending death threats to someone that dissed their favourite singer. Just super strange stuff.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#709

Post by Arubis »

voidUpdate wrote: post #199

> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much

You are responding to the original article, authored by John Gruber, who, last I checked, has been writing his own damn text multiple times every damn day for multiple decades as a primary vocation.

I’m willing to venture that Gruber is on the list of folks that get to hold the opinion choice of words matters.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#710

Post by unclebucknasty »

mbreese wrote: post #472
unclebucknasty wrote: post #423

Earlier quoted context omitted.

Simple version: In instances wherein the otherwise statistically chosen next word is a "toss-up", watermarking removes the randomness by imposing specific choices, determined by a key. This then becomes a detectable pattern when scanned with the key (stastically—detection itself is probabilistic). > use it to store arbitrary information No additional data is embedded. The range of available data is constrained by the…

From what I’ve read, they won’t be imposing specific choices, but using a different (biased) RNG for those “toss-up” choices. With enough sampling, you could detect if the RNG was biased or not.

This is what I meant by "imposing specific choices, determined by a key". Maybe "impose" or "specific" were too strong in my attempt to simplify?

I attempted to clarify that the impositions themselves are not deterministic, by indicating that the entire process is still probabilistic.

Maybe Anthropic's explanation is simple enough [0]:

>When watermarking is used, choices are still made at random, but the source of the randomness is different. Instead of using an arbitrary random number generator to pick the next word, watermarking uses the key and a few words that come before to settle what word the model should pick. That is, the words that Claude picks are still random, but now, one can check the sequence of words and see if it’s consistent with the choices Claude would make if it was using the key. If it is, one can assign a probability that the text was generated by Claude.

[0] https://www.anthropic.com/news/claude-text-watermark

Post reply on HN