Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

331–340 of 446 posts

Re: How Claude marks AI-generated content

#331

Post by andai »

jrflo wrote: post #300
Dilettante_ wrote: post #120

I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over t…

I think that false positives are inevitable due to the method of watermarking being embedded in the text itself. The output is intended to mimic human writing, therefore it's entirely conceivable that a human could by chance write text that contains the watermark. The odds may be extremely small, but it's not something you could ever guarantee.

I keep hearing how humans are thinking and writing more and more like AI.

I think in this case I think it's some kind of cryptographic signature smeared across the token IDs, so I don't think the risk is very high.

Re: How Claude marks AI-generated content

#333

Post by altmanaltman »

sfink wrote: post #303
mchusma wrote: post #230

Many good comments here. It’s somewhat common for me to voice record say a blog post of product updates, more like a ramble. Then have Claude clean it up. Then argue back and forth about certain things until it’s good, then make a final pass sometimes to change a few key words. This is incredibly different than pure ai text. Presumably it will show as ai generated here, even though I would argue it is not really. So…

Perhaps. But it seems like your beef is not with the presence of watermarking, it's with what people will use that watermarking for. You're not directly harmed by that blog post being labeled as AI generated. In a hypothetical (but unfortunately likely) world where everything has passed through an AI's digestive system, nobody would care. In the meantime, it is true that this takes something away from you. But it's s…

I would push back on the fact that nobody would care. It is clear that platforms are increasing creating AI-generated as a category and it will only get more precise over time. Platforms that have built trust/aura over what type of content they host will resort to this more as they get more flooded with ai generated low effort content. I recently wrote a blog on this actually: https://decodingvibes.com/blog/aura-and-the-backlash-against...

Re: How Claude marks AI-generated content

#334

Post by pram »

silversmith wrote: post #225
user43928 wrote: post #95

Earlier quoted context omitted.

I am wondering how that applies to newly generated code. Odd variable naming? Stylistic choices that are watermarked? Or as someone else noted further down in the comments, it could be more subtle: Between the first and second most likely choice, in certain positions it will consistently choose in a certain way.

Personal observation: Opus 5, over the last week, has started outputting A LOT more comments. Despite my global instructions being full of variations on "don't use comments unless absolutely necessary". I might be imagining things of course. But comments would be great fit for this use case.

Yes the length of comments Opus 5 leaves is exhausting. Not to mention it will insert info thats only relevant within the current session. I've just been deleting all of them lol

Re: How Claude marks AI-generated content

#335

Post by MattSayar »

mchusma wrote: post #230

Many good comments here. It’s somewhat common for me to voice record say a blog post of product updates, more like a ramble. Then have Claude clean it up. Then argue back and forth about certain things until it’s good, then make a final pass sometimes to change a few key words. This is incredibly different than pure ai text. Presumably it will show as ai generated here, even though I would argue it is not really. So…

I've done this before at work, and I feel true ownership of the output after this workflow. Moreso than when someone from a marketing team publishes a blog with the CEO's name as the author.

Re: How Claude marks AI-generated content

#336

Post by pessimizer »

I've been thinking that they have to be doing this. It seems like a fun problem, actually - all you're trying to encode is a 1-bit message within a text with the least amount of necessary changes possible, but in a way that arbitrary fragments will show it.

My intuition is that this would be very possible, in a way that makes false positives so unlikely as to be virtually nonexistent (at a certain fragment length.) Basically all you would be trying to do is to defeat people who would deliberately screw up the signal below the fragment length, and you would try to get that fragment length to at least the size that intentional obscuring of the signal would be obvious. I could see it being possible to detect even from non-contiguous fragments interspersed with noise.

It's just 1 bit, and you don't really care if a sentence or two is slop. I'd be surprised if a PhD interested in steganography couldn't come up with a good scheme in a week. It's a QR code.

What would be scary is if they could come up with a way to detect advice from Claude i.e. you get Claude to review your work as an editor, read the output, then as a result make non-verbatim changes, and that signal still gets through. If you could do that, you could do things like tell if a pundit speaking on television has read a particular Wikipedia page. Seems impossible, but LLMs seemed impossible.

edit: there are so many unimportant language choices; ones that are even hallmarks of AI use already, like the fact that it generally picks the mode. Not always picking the mode or picking at precise distances from the mode could hide signals without significantly affecting the quality of the content.

Re: How Claude marks AI-generated content

#337

Post by partiallypro »

My question is what's to stop Google or any competitor from using watermarks of Claude or OpenAI from degrading the rankings of sites that use it but ignore or even reward sites that use Gemini. Seems like an easy thing to do for competitors, and maybe an unforeseen side effect of these types of things or regulations.

Re: How Claude marks AI-generated content

#338

Post by subsistence234 »

philote wrote: post #247
nemomarx wrote: post #228

Earlier quoted context omitted.

This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it. I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can ofte…

That still might work better with older texts. As AI-generated text gets more prevalent, I'm guessing people will start subconsciously adopting AI writing styles.

LLMS aren't the only thing that has changed over time in the way texts are written.

if they used older texts as training data, to some extent pangram would just be an age classifier for writing style.

Re: How Claude marks AI-generated content

#339

Post by kccqzy »

DaiPlusPlus wrote: post #253
toxik wrote: post #143

Earlier quoted context omitted.

In program code? Unlikely, surely¡

That was actually the cause of an issue I had a couple of years ago: I had hand-typed JSON using my iPad into GitHub’s online text editor and Safari helpfully used “pretentious quotes” instead of "old-school quotes" - and the JSON library used by the program to read that file had relaxed parsing rules that accepted actual JS object literals without quoted property names; so the fancy-quotes were interpreted as part o…

I don’t doubt your experience, but many people are intimately aware of the use of proper Unicode quote characters, in any reasonable font they choose. For me, one of the first things I learned when using LaTeX is how the quotes are transformed from the source to the typeset document; since then I’ve become extremely sensitive to the kind of quotes I see.

Re: How Claude marks AI-generated content

#340

Post by jobigoud »

wtfwhateven wrote: post #235
sureMan6 wrote: post #127

Earlier quoted context omitted.

Maybe it has no false positive rate

Why would you say something so ridiculous?

I think they mean it like this: imagine you ask me a random number sequence. I give you a random number sequence. Little did you know, I used a very specific PRNG to generate it, so later I can prove with certainty that your number was generated by me, and you can't say you came up with it yourself.

There is no room for false positive here in the same way you can't randomly find a collision in a hash function if it's strong enough. Like the rate is so infinitesimal that it is effectively zero.

Now replace random number sequence with prompted string of words. And instead of using the PRNG on every word I use it every n words. If the generated text is sufficiently long I can tell by matching the expected deterministic pattern.

You can defeat it by changing the words yourself and triggering a false negative but there isn't really any room for a false positive if the text is long enough and the pattern matches perfectly. If the pattern doesn't match then I can compute a probability.

Post reply on HN