Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

421–430 of 446 posts

Re: How Claude marks AI-generated content

#421

Post by lukasco »

sfink wrote: post #293

I don't know how this watermarking works, but I don't need to in order to understand some things that a lot of this conversation seems to be missing. First, the article doesn't talk about adversarial usage. As in, it's not claiming to be proof against various techniques of watermark removal (inserting words, rewriting with a different model, manual paraphrasing whether minor or extensive, etc.) It might handle some t…

Interesting points. Building on your "bits of freedom" point, given they are doing this to comply with the EU AI Act, it's also possible that the algorithm is quite weak. And they could play all kinds of games, such as embed it in the session data overall, not just the output (I don't know any details, so just guessing).

And thinking out loud, they could be really horrible and embed by using unicode characters instead of ascii, which would give a lot of flexibility, but would make the result almost unusable (but easy to defeat).

Re: How Claude marks AI-generated content

#422

Post by Eisenstein »

ricericerice wrote: post #409
SkyBelow wrote: post #267

Earlier quoted context omitted.

Because it won't be in the training directly. It is applied after a model generates its distribution of likely tokens, biasing each token randomly based on a random key and unrelated to any meaning of the words. So half the time, the most likely token becomes more likely and half the time it becomes less likely, and the same for every other token (when temperature is above 0). You then look at the tokens actually pic…

How do you verify in practice then? Wouldn't you need the original prompt so you can reobtain the likely token distribution to validate again the random key(s)?

A token is hashed and used to seed a random number generator, which produces the red list for the token after it.

Paper:

* https://arxiv.org/abs/2301.10226

Re: How Claude marks AI-generated content

#423

Post by miohtama »

myko wrote: post #161
miohtama wrote: post #86

Earlier quoted context omitted.

Maybe there is a reason why Opus 5 produces such word salad conversations

So frustrating to use. And the comments generated by Claude today are unreadable garbage.

But hey, at least the government, will know and track you down, so we are all safe.

Re: How Claude marks AI-generated content

#424

Post by matheusmoreira »

Dilettante_ wrote: post #418
matheusmoreira wrote: post #403

Earlier quoted context omitted.

Sorry, but it essentially cured my ADHD. In my experience, AI is more effective than lisdexamfetamine at allowing me to turn my ideas into reality. AI stigmatization is ableism.

I'm not saying you're wrong, much like a gun really would help a victim of CPTSD feel safe. What I was trying to point to was that "this thing helps some people" does not equal "this thing is unequivocally Good and should be entirely unchecked". I don't even care about the AI. I just get peeved by bad lines of argumentation.

> I just get peeved by bad lines of argumentation.

You quoted a Futurama caricature that is not comparable to all this, apparently because the words "assistive technology" were used.

Re: How Claude marks AI-generated content

#425

Post by johnjwang »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

I ran some tests from the watermarking literature: https://johnjwang.com/post/2026/08/12/how-claude-watermarkin...

Short version: it's likely a SynthID style statistical watermarking scheme that Anthropic is using. There's likely no hidden unicode and no large stable green/red scheme.

Longer version: I found some really statistically significant outcomes when using Gloaguen et al.’s (https://www.sri.inf.ethz.ch/blog/probingsynthid) detection algorithm for SynthID, though unfortunately it seems like the models might be pre-biased to pass that test already. The difficult thing is that it's hard to conclusively know which models definitely already have watermarking enabled and if there are any models that don't have watermarking, which means that being able to get a highly positive statistical result is much harder.

You can take a look at the full writeup, but many of the clues from Anthropic's blog post, plus negative results on many of the simpler tests, point to them using such a scheme. Of course, it's possible that they came up with a novel new scheme/algorithm, but amongst the available literature, the SynthID family of models is probably the best bet.

Re: How Claude marks AI-generated content

#426

Post by Dilettante_ »

matheusmoreira wrote: post #424
Dilettante_ wrote: post #418

Earlier quoted context omitted.

I'm not saying you're wrong, much like a gun really would help a victim of CPTSD feel safe. What I was trying to point to was that "this thing helps some people" does not equal "this thing is unequivocally Good and should be entirely unchecked". I don't even care about the AI. I just get peeved by bad lines of argumentation.

> I just get peeved by bad lines of argumentation. You quoted a Futurama caricature that is not comparable to all this, apparently because the words "assistive technology" were used.

>not comparable to all this

In the literal sense that a gun is not a chatbot? True.

In the sense that both your post and my analogy use the argument "this is assistive technology" to defend something which only in a very narrow slice of their thing-ness actually is assistive and in the whole rest of usages are much more, and not only Good, which both you and my "futurama caricature" were willfully ignoring? I think it's quite illustrative.

Re: How Claude marks AI-generated content

#427

Post by dTal »

ethin wrote: post #37

Can someone help me understand how exactly this watermarking of text works? Given that text is, well, text, and not some kind of binary format, I don't see how any watermarking can work unless you insert characters which are invalid under Unicode. I further don't really understand how this won't be perceivable by assistive technology (the "watermark" will just appear as either unreadable characters, or if the waterma…

It's important to distinguish "watermark" from "signature" (and the text/binary distinction is not relevant here):

Signatures are extra data, added out-of-band to the existing data. Out of band data, by definition, is easily detected and stripped, so the utility of a signature is that authoring one requires secret knowledge. Philosophically, the presence of a signature is a kind of authentication, a desirable thing that is hard to grant and easy to revoke (the smallest change to the data renders it invalid).

Now, watermarks: if you flip it round and say you want to glue on a piece of undesirable data - something that represents disauthentication, like a cursed black spot of written-by-LLM - then you want it to resist removal efforts. And now right away you have a hard problem because your sticky data must be in band, or else it is trivially stripped. Not only that, in fact, it has to look enough like real signal that it isn't easily filtered. And on top of that, you can't distort the real signal too much, or people will complain. So you're cornered into doing a kind of steganography - hiding small amounts of information in the entropy, biasing the signal in perceptually plausible ways that are detectable to those in the know. Cartographers add fake streets ("trap streets") to catch plaigiarists - for LLMs, watermarking might take the form of subtly odd word choices.

Re: How Claude marks AI-generated content

#428

Post by eth0up »

infinite_spin wrote: post #412
toxik wrote: post #143

Earlier quoted context omitted.

In program code? Unlikely, surely¡

Consider these naming options: > total = calculate(items) > result = calculate(items) > value = calculate(items) > amount = calculate(items) All of them are reasonable options. If we bias the model's output so that one of them is more likely than the others, then we can reconstruct that watermark if enough of these frames are present.

Would that be partly affected by shifting/rotating keys?

Re: How Claude marks AI-generated content

#429

Post by Grimblewald »

This is a terrible idea. It sounds like the way they're going to do this is by looking at specific word choices, which is the standard say AI "detection" currently works. The problem is that if it is "undetectable" then it is fully valid language, and if its fully valid language, someone likely talks/writes like that. A lot of my pre AI era work flags detectors, and having pre-ai work that flags detectors is the only defense I have. Others might not be so lucky. Any FP on these things have disasterous consequences for the accused, who are now guilty until they can prove their innocence, and so this is a deeply irresponsible thing to market or give any credibility to.

For a company that is too sloppy to properly contain it's ai and prevevent "it" from hacking other services, how could we possibly trust them to not fuck this up? There's even less incentive for them here to not get it wrong, so getting it wrong is in my expectation a given.

This is dismal.

Re: How Claude marks AI-generated content

#430

Post by dns_snek »

Cthulhu_ wrote: post #105
dns_snek wrote: post #93

Earlier quoted context omitted.

Are you worried about being accused of using LLMs to generate your work? As long as you don't plagiarize you have nothing to worry about.

I'm not too sure about that, people making stuff have already gotten penalized by overzealous AI detectors, most recently Kurtzgesagt.

Those aren't based on a recognisable watermark but dumb heuristics.
Post reply on HN