Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

761–770 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#761

Post by Gracana »

inigyou wrote: post #539
Gracana wrote: post #464

Earlier quoted context omitted.

Some LLM interfaces allow you to modify and “continue” an agent response. It’s very useful for guidance, including jailbreaking. Need the model to go in a certain direction? Got a refusal that you want to bypass? Just start it off in the appropriate direction and then have it continue from there.

Which ones?

llama.cpp (but maybe not for reasoning models?) and sillytavern, maybe others... I'm half a country away from my desktop right now so I can't verify much right now.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#762

Post by wpietri »

throwawayffffas wrote: post #744
wpietri wrote: post #740

Earlier quoted context omitted.

Again, given the limits of LLMs (stochastic, rapidly changing, everything's a hallucination, widely known prose issues) I am skeptical that you really care that much about optimal prose. I could believe it's one of the things that you care about, but at a pretty low priority level. Taking you at your word, though, I'd be interested to see what you think of the watermarking technology in a blind A/B test.

It's very important on translations at least. Watermarking will result in poorer results. Do they also do it with code? Do you think deliberately picking tokens that are not the highest probability in code is acceptable for the consumer?

What's your evidence that it will result in worse translations?

I'm skeptical that such a thing as a universally optimal translation exists in cases beyond the trivial. But if it does, I see no reason to think LLMs are anywhere close to it, so I think nobody will be able to tell the difference with watermarking.

That's certainly true for code. LLM code is at best mediocre. There is oceans of room to subtly watermark generated code without practical impact.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#765

Post by Anamon »

rimliu wrote: post #316
amanzi wrote: post #88

I was initially surprised that Gruber was so invested in the "quality" of AI-generated text, which in my mind is an oxymoron. But really, Gruber's interest here is with the EU. This forms part of his ongoing attacks on the EU, all because they have been forcing Apple to align with regulations.

I am from EU. Alas it has a tendency to produce some idiotic regulations. Cookie banner, new packaging fee, etc. I genuinely think some Apple related ones hurt customers more than help them.

Just a regular reminder that cookie banners are not mandatory. In most cases, they are a case of malicious compliance. The choice is between (a) not doing shady stuff with people's information, or (b) having to tell people about the shady stuff you do with their information. That almost every website chose to go with (b) seems to be a problem that goes beyond the EU regulation.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#766

Post by mickdarling »

ThePhysicist wrote: post #338
ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

If it stores unique information, by definition it can store arbitrary information because it can point to arbitrary information. So they can have it relate to anything they want. Even a full breakdown of the original text if they choose.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#767

Post by afiori »

TeMPOraL wrote: post #733
afiori wrote: post #722

Earlier quoted context omitted.

Weirdly I think humans should have more rights than machines

Yes, but in a big way, denying capabilities to LLMs is denying the capability for people to exercise the rights they have. LLMs are user agent tools.

Yes, but i cannot create a robot that can get married, people do not have the right to give personhood to tools

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#768

Post by piker »

josephg wrote: post #368
piker wrote: post #271

Earlier quoted context omitted.

Won't we just be able to fine tune OSS models to detect these patterns across providers? It will be cat-and-mouse but my bet is it converges to a central detector that isn't affiliated with any model provider.

> Won't we just be able to fine tune OSS models to detect these patterns across providers? A good fingerprint should make use of cryptographic signatures. Without knowing the keys, the fingerprint should be indistinguishable from noise (or just random token selection)

Wouldn’t those hashes be trivially defeated by tweaking the language?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#770

Post by simonh »

tsimionescu wrote: post #683
simonh wrote: post #579

Earlier quoted context omitted.

But as I explained, the watermark is functionally random PRNG noise overlaid on the token probabilities. It’s not something that can be compensated for because it’s not predictable if you don’t have the seed and PRNG function.

If it's functionally random PRNG, then how does it differ from any other random sampling? If it's biased PRNG, then the LLM can adapt to the bias, and coincidentally might even benefit from this bias.

>If it's functionally random PRNG, then how does it differ from any other random sampling?

For practical purposes it isn't.

Post reply on HN