Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

51–60 of 446 posts

Re: How Claude marks AI-generated content

#51

Post by phainopepla2 »

w_for_wumbo wrote: post #34
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

What happens if someone handwrites a Claude output, then someone uses that handwritten text as a reference. Now you've got a watermarked idea which may have no direct linkage to the usage of Claude.

How is that different from referencing digital text that someone copied and pasted from Claude?

Re: How Claude marks AI-generated content

#54

Post by cassianoleal »

benrow wrote: post #36

I've heard that this kind of watermarking process works by biassing the statistical sampling towards a partition of the set of possible next tokens (red set and green set), at each position. It might only be a slight nudge each time, but over a sequence of tokens, the likelihood of repeating the bias by chance is increasingly improbable. The bias is different for each position and follows a defined RNG, seeded someho…

> a defined RNG, seeded somehow predictably

So, an NG?

Re: How Claude marks AI-generated content

#55

Post by m-chrzan »

benrow wrote: post #36

I've heard that this kind of watermarking process works by biassing the statistical sampling towards a partition of the set of possible next tokens (red set and green set), at each position. It might only be a slight nudge each time, but over a sequence of tokens, the likelihood of repeating the bias by chance is increasingly improbable. The bias is different for each position and follows a defined RNG, seeded someho…

There's a computerphile video (https://www.youtube.com/watch?v=XZJc1p6RE78) with Dr. Mark Pound explaining a paper by John Kirchenbauer, Jonas Geiping et al. (https://arxiv.org/abs/2301.10226) that described a method for watermarking LLM output like this. It's not directly stated anywhere in the Claude support article that this is what they're using, but the properties of the watermark described seem to point to this method.

Re: How Claude marks AI-generated content

#57

Post by hannasanarion »

akozak wrote: post #30
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Most likely this method https://arxiv.org/pdf/2301.10226 (EDIT: and Google's SynthID paper which builds on it https://www.nature.com/articles/s41586-024-08025-4 )

That "just add a constant to the green logits" as a fix to the entropy problem is so elegant I love it.

Re: How Claude marks AI-generated content

#58

Post by hparadiz »

cassianoleal wrote: post #56
drnick1 wrote: post #49

Seems like an awful idea. I hope that that "watermark" will soon be discovered, reverse-engineered, and that tools to remove it will appear.

I hope all models adopt it.

Other models will end up diffusing it and making the signal indeterministic and irrelevant.

Re: How Claude marks AI-generated content

#59

Post by siva7 »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

I can tell you how: Claude produces a huge wall of text with jargon ridden bullshit and invented terms no human subject matter expert would seriously use and overuse.

Re: How Claude marks AI-generated content

#60

Post by pixl97 »

lorenzohess wrote: post #4

> Generated text will carry embedded watermarks, and generated files will include digitally signed provenance metadata where supported. This should make it easier to catch cheaters who use Claude, right? Unless everyone runs their artifacts through some watermark and metadata sanitizer?

Text is too low bandwidth to classify reliably without lots of false positives. Especially as people start talking like LLMs.
Post reply on HN