Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

21–30 of 446 posts

Re: How Claude marks AI-generated content

#21

Post by simonw »

An interesting factor of this is competition.

If Claude was the only model family they could ship a change like this and users who want to cheat (or don't like watermarks for other reasons) would just have to put up with it.

In a world with many different competing models, the risk of losing customers to other providers over this is much more real.

Maybe they've looked at the numbers and the portion of people who clearly use Claude to cheat on examples etc is so tiny that losing them to other providers isn't a problem?

Re: How Claude marks AI-generated content

#22

Post by kingstnap »

pjm331 wrote: post #17
aabhay wrote: post #6

I have had a hunch for a while now that (in addition to these tools), Anthropic has actually leaned in to Claude's distinctive manner of writing since it makes the text more obviously AI generated and thus less susceptible to misuse. That's not necessarily the same thing as a markov-style fingerprint but it could be a correlated factor.

I had a similar thought but I assumed they leaned in because it improved performance on coding or something like that

It could also partly be a byproduct of examples of claude writing being in the dataset, which of course anthropic has lots and lots of and they do train on.

Re: How Claude marks AI-generated content

#24

Post by gajus »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Most likely watermark will be proportional to the input/output ratio, i.e. if you input a long document and ask to make edits, it will not attempt to watermark it. On the other hand, if you provide a tweet and ask it to write an article, that will include watermark. Just a guess (and yes, it feels flawed)

Re: How Claude marks AI-generated content

#26

Post by stabbles »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

It will just thread some load-bearing seams through the paragraphs.

Re: How Claude marks AI-generated content

#27

Post by gajus »

simonw wrote: post #21

An interesting factor of this is competition. If Claude was the only model family they could ship a change like this and users who want to cheat (or don't like watermarks for other reasons) would just have to put up with it. In a world with many different competing models, the risk of losing customers to other providers over this is much more real. Maybe they've looked at the numbers and the portion of people who cle…

There are already small models trained specifically to prevent statistical detection, e.g., https://huggingface.co/kalpeshk2011/dipper-paraphraser-xxl

I guess whoever is the policy maker is assuming that some protection is better than none and that most people will not reach for such tools.

Re: How Claude marks AI-generated content

#28

Post by Computer0 »

toufka wrote: post #18
travisgriggs wrote: post #3

Earlier quoted context omitted.

I would love to see what this looks like in practice. Especially in generated code. I assume this is more than insertion of non visible special unicode whitespace characters, but more in the pattern of the text content itself?

I'm guessing - probably some textual variation on Benford's law? [1]. Trivial for compute, painful for a human. - "Ensure distribution of vowels is in >99th percentile of human work" - "Ensure the distribution of the letter "s" is within 99th percentile of human work" - "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters. - "Ensure there is a cross-linguistic 'typo' (co…

I would hate to have any of these rules effecting my output

Re: How Claude marks AI-generated content

#29

Post by baq »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

> I'd like to know a lot more about how that works.

Count load-bearing words using two different algorithms in a belt-and-braces fashion

Re: How Claude marks AI-generated content

#30

Post by akozak »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Most likely this method https://arxiv.org/pdf/2301.10226 (EDIT: and Google's SynthID paper which builds on it https://www.nature.com/articles/s41586-024-08025-4)
Post reply on HN