Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

381–390 of 446 posts

Re: How Claude marks AI-generated content

#381

Post by wrsh07 »

Groxx wrote: post #350
denverllc wrote: post #326

Earlier quoted context omitted.

I’ve always wondered how this works when we only observe the final output and not the internal state that’s used to generate the output. The LLM presumably generates f(input, RNG) but we only can observe f(RNG).

Since they do have the input, they could probably just store checksums at each step... ... though I'm not sure why that would be preferable over a coarse rolling checksum over all of the output. Seems like that wouldn't influence output, would be equally imperceptible, and probably easier to calculate (compared to "hash seed times running all LLMs supported times number of RNG algorithms, to see if output matches").…

No you don't need to do that, the prng is detectable if you know what bias to look for and have the key

Re: How Claude marks AI-generated content

#382

Post by wrsh07 »

teravor wrote: post #374
wrsh07 wrote: post #221

Earlier quoted context omitted.

Scott Aaronson talks about his project at OpenAI to do this^ You can carefully select which pseudorandom number generator (prng) you use to be able to id text of a certain length. I expect there is some performance characteristic you have to manage since you're doing this on every inference, but once you do that it doesn't change the output in any meaningful way (the prng is still a statistically valid prng, it just…

this would be a very heavy watermark application. there are many simpler methods, for example you can have a tiny windowed transformer operating on the output text and all you do is alter certain words (that don't change meanings) to maximize its surprise. the tiny language model will have a special training regime to build up a somewhat unique view of the language. we are talking about a 0.5 bit watermark here (exis…

That's actually much worse because it fundamentally changes the output, whereas this doesn't change the output, it just changed the prng

Re: How Claude marks AI-generated content

#383

Post by KronisLV »

> Claude models launched in the EU

Could we get an Anthropic subscription for Claude Code with data residency in the EU, so we don't get robbed blind by AWS Bedrock et al., but can have a monthly subscription like with the regular US option?

Re: How Claude marks AI-generated content

#385

Post by wrsh07 »

basch wrote: post #237
wrsh07 wrote: post #214

Earlier quoted context omitted.

Somewhat trivially, if I ask Claude to transcribe an image and then check if that transcription is ai generated it will likely say yes. Many users are not smart enough to realize that the transcription step is where the ai (watermarks) were necessarily injected.

How is a perfect transcription of an image watermarked?

It depends on how it does watermarking!!

Note, there are many ways to represent words visually on computers that look identical

Re: How Claude marks AI-generated content

#386

Post by basch »

wrsh07 wrote: post #385
basch wrote: post #237

Earlier quoted context omitted.

How is a perfect transcription of an image watermarked?

It depends on how it does watermarking!! Note, there are many ways to represent words visually on computers that look identical

If they were substituting glyphs for identical ones people would be able to reverse engineer it.

Theres no way that’s what they are doing.

Re: How Claude marks AI-generated content

#387

Post by breezybottom »

someguynamedq wrote: post #248
breezybottom wrote: post #197

Earlier quoted context omitted.

People have been posting some variant of this comment for three years, and it's no more true today. Ever notice that the "prompt engineer" career hasn't materialized?

Prompt engineer is a requirement within every serious job now, not a job in itself

I've never known any firefighters to prompt engineer a blaze, but perhaps you don't consider that a "serious" job.

Re: How Claude marks AI-generated content

#388

Post by andai »

tiahura wrote: post #344
andai wrote: post #330

If I understand correctly, this means that any text with the "watermark" is legally uncopyrightable, including code. Relevant comment from a few days ago: https://news.ycombinator.com/item?id=49203613

Not an copyright attorney, but color printers have watermarks. That's never been an obstacle.

AI generated content is not copyrightable, as far as I understood. A reliable watermark is a reliable indicator that a work does not fall under copyright.

For a repo, I don't know what that means. Only the AI generated lines are public domain?

Re: How Claude marks AI-generated content

#389

Post by derefr »

Dilettante_ wrote: post #327
whack wrote: post #283

Earlier quoted context omitted.

> But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated Are you talking about pieces that were fully human-written with zero AI editing/rewriting etc? If so, what makes you think that false positives will happen there? They aren't looking for "writing styles" or emdashes etc. They are using watermarks and metadata…

The former. I'm not sure what you mean by metadata, but my expectation was that anything that Claude could put into the plaintext to identify itself may plausibly also accidentally be produced by [a million monkeys on typewriters/one in a million human writers], since in the end, the writing is using the same language and symbols that humans use. How unique could the LLM possibly make it while still retaining its use…

> How unique could the LLM possibly make it while still retaining its usefulness?

They could be doing invisible and vaguely-harmless Unicode stuff. Insertion of zero-width joiners and non-joiners, replacement of regular spaces with non-breaking spaces, building spaces from multiple hairline spaces, intentional use of non-NFC-normalized codepoint sequences for accented characters, etc.

Text with all this junk in it still reads the same; it just might wrap a little strangely, or not byte-match / collate correctly in a database (and Anthropic has never made a guarantee that their models would be capable of emitting text with these properties, so that’s fine.)

And, importantly, no regular text or document editor would insert these things (especially in the useless places you could insert them for watermarking.) You only really see them in text that’s been explicitly typeset for a specific layout (e.g. in text-containing SVGs, website mastheads, or game HUDs) or for print publication.

Of course, if this is the technique they end up using, then it’s very simple to strip it out by canonicalizing the text (i.e. Unicode-normalizing it + stripping out invisible layout characters + replacing “weird spaces” with regular ones, etc. Essentially the same thing many sites already do to user-generated content to prevent users from using Unicode features to break the page’s layout.

Re: How Claude marks AI-generated content

#390

Post by andai »

luxuryballs wrote: post #343
andai wrote: post #330

If I understand correctly, this means that any text with the "watermark" is legally uncopyrightable, including code. Relevant comment from a few days ago: https://news.ycombinator.com/item?id=49203613

what if the output is downstream from copyrightable work? wouldn't the LLM touching it wash that off if this was the metric used?

Well, all major LLMs have been trained on copyrighted material, so effectively all LLM output is downstream of copyrighted work.

I guess it's a bit weird. An LLM can recite copyrighted material verbatim from training data (they had to work hard to get them to stop doing that, and they haven't been entirely successful). But LLM outputs are public domain. (Except when they are a verbatim reproduction of a copyrighted work.) I'm not sure where you draw that line when it's not verbatim...

Post reply on HN