Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

61–70 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#61

Post by LoganDark »

walrus01 wrote: post #46
LoganDark wrote: post #40

Earlier quoted context omitted.

That is not what that means. Generally, precise word choice requires more than autocomplete. Larger models simulate this with hidden layers.

Excessively precise word choice does not result in something that looks like content written by, or palatable to humans. It looks like you gave a high school 12 grade student a science paper and told them to apply a thesaurus to at least one word in every sentence and replace it with something else.

There is a difference between precise word choice and concise word choice. You can be precisely accessible the same as you can be concisely terse.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#62

Post by brookst »

reader9274 wrote: post #48
levocardia wrote: post #26

Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).

"Smart"? Have you read his writings in the last decade? It's all nonsense, which is why I stopped reading circa 2018

I think he’s still generally good on business, UX, and hardware design. That’s all subjective and taste I suppose, but his taste works for me.

On deeper tech stuff, like this utterly nonsensical misunderstanding of watermarks… yeah, classic case of a guy who is smart, and has lost the ability to realize when they’re not knowledgeable in a domain.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#65

Post by roywiggins »

it serves to show just how little regard the people behind these generated-text fingerprinting schemes have for the actual craft of writing.

LLMs have never been the place I've thought to expect any commitment to the craft of writing, to be fair.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#66

Post by Gigachad »

Imnimo wrote: post #51

>I want any LLM I use to choose the very best, most precise words at every single decision point. Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing? This entire article just seems so detached from the basics of how LLMs work.

I think the author is just mad people will be able to detect and filter out their AI slop writing in the future.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#68

Post by brookst »

npilk wrote: post #19
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…

This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#69

Post by brookst »

tomrod wrote: post #52
mholm wrote: post #39

Earlier quoted context omitted.

This was true in the ChatGPT era. Now we're in a world with reasoning tokens, where a model can thoroughly plan out the response it wants to make. If anything, it makes the style worse.

Isn't this just chain-of-thought though, doing the same thing multiple times without necessarily defining one path?

Chain of thought attempts to guide a single forward pass.

Reasoning tokens with tool calling tell the model to loop on a one phase of a question and call a tool to indications completion when done.

Related, but not the same thing.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#70

Post by beering »

avaer wrote: post #49
syrrim wrote: post #9

> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…

That's missing the point. It's the distribution that's the "best", not the tokens. Then Anthropic comes in and makes the distribution something other than the best. The only saving grace is that Anthropic says it's not that bad. Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that ba…

The watermark doesn’t change the distribution, only per-token selection. I think not understanding that is the source of most people’s FUD.
Post reply on HN