Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

601–610 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#601

Post by dofm »

breezybottom wrote: post #420
dofm wrote: post #336

Earlier quoted context omitted.

Why? I really don't understand this. Why can't a tech writer take a deep but neutral intellectual interest in something? Isn't it important that some do? Do people have to be stakeholders or clearly on one given team, pro- or anti-, for their opinion to matter? Is it that tribal? It seems fully logical to me that someone who writes for a living (who, as it happens, developed the very markup language LLMs use for ever…

If he writes then he should have no investment in LLMs. Humans have been able to write for thousands of years.

You've somewhat mischaracterised what I have said.

I suggested he as a tech writer has reason to be invested in understanding how they work. It's not really a sustainable position to not understand, is it?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#604

Post by summarybot »

inigyou wrote: post #536
summarybot wrote: post #508

Earlier quoted context omitted.

Copyright law was updated in a very helpful way in the last twenty years sometime so that as soon as you post something to the internet you have copyright. If you need a citation don't hesitate to ask someone else.

So you can just say some text is copyright to you and it is? Here is some text that is copyright to me. As you infringed my copyright, please pay my $5000 license fee for every user who has read it: > Copyright law was updated in a very helpful way in the last twenty years sometime so that as soon as you post something to the internet you have copyright. If you need a citation don't hesitate to ask someone else.

Rather casuistic take, there.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#605

Post by cocoa19 »

ThePhysicist wrote: post #338
ghrl wrote: post #246

My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

Removing the watermark usefulness depends on your use case.

If you care to avoid detection, yes, it is useful. If you care about the best possible sequence of words, then the damage is already done once watermarked.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#606

Post by bluegatty »

This was nice but without hard evidence. I'll bet if nobody were to have said anything, nobody would nave noticed. That doesn't make it right, but it might be less intelligible and intrusive than the claims here.

Also this:

"> I want any LLM I use to choose the very best, most precise words at every single decision point."

I don't think the author quite appreciates the level of randomness here.

This is more subtle then Claude changing prose.

Again - I suggest that the author would have to run a test on themselves to determine if they can actually find a difference.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#607

Post by wrsh07 »

vintermann wrote: post #379
unbelievr wrote: post #361

Earlier quoted context omitted.

The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…

A way to think of it is that any time the model faces a choice, it leaks some fractional bits of information. Instead of making the choice randomly, you can put information in those bits. But unless you know the prompt, you don't fully know which choice the model faces. Surely a lot of coding space is wasted compensating for that uncertainty. Exotic prompts ("Use no more than five E's in four consecutive words anywhe…

The scheme that Scott Aaronson describes essentially uses a specific prng, and you can then check a certain function with relatively few tokens to get a sense of whether or not a model using that scheme generated the text

A few caveats: you need to know the key to the function (used when generating the text) and you need to know the bias it would introduce

The point is that you do not need to know the full prefix, just a modest sample set of contiguous tokens

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#608

Post by regularfry »

simonh wrote: post #325
regularfry wrote: post #264

Earlier quoted context omitted.

It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator. We already use an RNG at inference precisely because it leads to higher-quality output . Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user. Fundamentally the article is railing against --tem…

>It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator. You can't as a user tell by how much the quality of the output was degraded. True. >We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness…

What you're arguing for is that the deviation from true randomness is worse than the PRNGs that are already used, and that the deviation is anticorrelated with some notion of quality. You need both. I don't think you've got either.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#609

Post by sfink »

I agree with the substance of this article, and disagree with the author's reaction to it.

The part I agree with:

It is true that watermarking can be done by "just" swapping one PRNG for another, and it is even true that with today's LLMs, it is possible that this will not degrade the output. But it has a cost, and as things improve, that cost will matter. You are intentionally reducing the degrees of freedom in the output, and using those bits of entropy for a purpose that does not improve the quality. If you maximize your tradeoff of bits for quality, those extra bits lower the ceiling of what's possible. It's a very simple information theoretic argument, and the only plausible argument against it (that we're using those bits so inefficiently now that the new PRNG is no worse than the old) only holds in the short term.

I also agree that having TOS that forbid removing the watermarking is deeply, deeply problematic. Hell, the whole essay is well-written and persuasive, and gives good reasons why this is a poor approach.

What I disagree with, and the reason for this comment, is the entitlement.

> The idea that anything other than my needs should factor into the generation of text for me is patently offensive.

This attitude is what is patently offensive for me. This is the argument that the world is beholden to my interests. It says that worrying about negative externalities is immoral. It's another form of certain people being above the law, shareholder profit maximization über alles, might makes right, we have to do it or someone else will, "we just help people connect", {code,a gun,roofie} is just a tool.

So I agree that the watermarking has a cost. But you can't leave out that it is an attempt to reduce negative externalities of AI. Whether it's a realistic or worthwhile attempt is a whole other debate (and Gruber does a good job of debating just that in the latter part of the essay), but saying that the user's needs are the only thing that should ever be considered is reprehensible.

Post reply on HN