Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

71–80 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#71

Post by dofm »

Imnimo wrote: post #51

>I want any LLM I use to choose the very best, most precise words at every single decision point. Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? This entire article just seems so detached from the basics of how LLMs work.

> Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing?

No and no. I am not sure I agree with his point but I know he is not ill-informed on either of these points, because I mentioned them to him a couple of days ago.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#72

Post by brookst »

avaer wrote: post #49
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

That's missing the point. It's the distribution that's the "best", not the tokens. Then Anthropic comes in and makes the distribution something other than the best. The only saving grace is that Anthropic says it's not that bad. Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that ba…

How do you know you picked the singular “best” set of tokens in your comment here?

Could it have been equal or better with slight variations in wording?

The slipper slop argument is too lazy to address directly. Argue A is bad because A, not because A might become B and you’ve got good arguments against B.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#73

Post by Art9681 »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

If this is true then the probability of the detection tools flagging completely human generated text as AI generated is non-trivial. Let's say I write a completely original piece and the detection tool says there is a 36% probability it was generated with Claude. What then? Now it's up to the person looking at the score to cast a subjective judgement. Maybe to me, anything over 25% is unacceptable. Maybe to someone else, it must cross over the 50% threshold. This is the problem.

Cognitive surrender.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#74

Post by brookst »

dofm wrote: post #71
Imnimo wrote: post #51

>I want any LLM I use to choose the very best, most precise words at every single decision point. Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? This entire article just seems so detached from the basics of how LLMs work.

> Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? No and no. I am not sure I agree with his point but I know he is not ill-informed on either of these points, because I mentioned them to him a couple of days ago.

It didn’t take, apparently.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#75

Post by egypturnash »

LLMs are already perversions of writing, so what else is new. Oh no, the over-long circumlocution generated by three autocorrects in a trenchcoat might be slightly longer because of this and maybe people will start noticing the subtle rhythms of vaguely peculiar word choices as yet another cue that you are wasting their time with machine-generated wordslop, what a terrible fate. Your long rambling walls of machine-waffling might be 37.05% longer than they need to be instead of the mere 36.58% longer they are now.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#77

Post by dofm »

Gigachad wrote: post #66
Imnimo wrote: post #51

>I want any LLM I use to choose the very best, most precise words at every single decision point. Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? This entire article just seems so detached from the basics of how LLMs work.

I think the author is just mad people will be able to detect and filter out their AI slop writing in the future.

This is not it, no. He is not using AI and it is not I think remotely in his nature to surrender that control. He is engaging with this on principle. Again I am not sure I agree with him, but then it’s a hypothetical because I am not going to get an LLM to write for me either.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#78

Post by wpietri »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

It seems to me like he started out mad and looked to justify it.

I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists.

If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experiment harness and rated a few hundred blinded examples and found a measurable difference.

But getting this upset in advance of any demonstrated problem? It really seems to me like the point isn't the point

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#80

Post by conartist6 »

levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

I couldn't be happier that people are mad about it. To quote Calvin, "nothing helps a bad mood like spreading it around"
Post reply on HN