Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

441–446 of 446 posts

Re: How Claude marks AI-generated content

#441

Post by lovetocode »

Der_Einzige wrote: post #40
toufka wrote: post #18

Earlier quoted context omitted.

I'm guessing - probably some textual variation on Benford's law? [1]. Trivial for compute, painful for a human. - "Ensure distribution of vowels is in >99th percentile of human work" - "Ensure the distribution of the letter "s" is within 99th percentile of human work" - "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters. - "Ensure there is a cross-linguistic 'typo' (co…

Models can't reliably follow instructions involving their own logprobs unless they can take agentic control and use quite sophisticated dynamic grammars/structures/constraints to force this behavior in one shot (which can be slow and the dynamic grammar modification feature isn't supported in closed model APIs for safety reasons) or repeated attempts at rewriting which is expensive/slow. Yes they can do this, but it'…

Well put. I wonder if this will inevitably increase usage costs because the model has to do more work to find a solution that meets watermarking standards.

Re: How Claude marks AI-generated content

#442

Post by platinumrad »

dns_snek wrote: post #431
platinumrad wrote: post #126

Earlier quoted context omitted.

You can't make a blanket statement like this without knowing how the watermark is implemented.

Why not? I'm assuming that the watermark detector won't have plausible false positives on human-written text otherwise it won't have much merit to begin with. If a detector flags something in your text and you've properly attributed that text to another author then what is there to worry about?

If you take it as given that the watermark detector won't have "plausible false positives" then, sure, your argument becomes trivial. I'm not taking that as given.

Re: How Claude marks AI-generated content

#443

Post by sisyphus15 »

thunfischtoast wrote: post #92
COAGULOPATH wrote: post #80

Earlier quoted context omitted.

>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…

They still need to choose when to do that though. When I prompt the program to e.g. alter a bash script in a specific way or to recite a longer known text it can't go round and randomly exchange tokens. It has to somehow define what is a simple repeated text from a different origin and what is a novel generation.

You are aware that Claude already doesn't choose the most probable token, right? That's literally what the temperature parameter means. It picks tokens at random (from the list of most probable next tokens), increasingly so as the temperature goes up. This has always been the case. And now, with the watermarking, it will simply go from random to pseudo-random, adding some patterning.

There isn't any effect on the quality or precision of the output. Nothing changes in practice.

Re: How Claude marks AI-generated content

#444

Post by dns_snek »

platinumrad wrote: post #442
dns_snek wrote: post #431

Earlier quoted context omitted.

Why not? I'm assuming that the watermark detector won't have plausible false positives on human-written text otherwise it won't have much merit to begin with. If a detector flags something in your text and you've properly attributed that text to another author then what is there to worry about?

If you take it as given that the watermark detector won't have "plausible false positives" then, sure, your argument becomes trivial. I'm not taking that as given.

I'm not taking anything as given. If it suffers from false positives then you have nothing to worry about because the signal won't have any merit -- just like existing AI detectors today. If it doesn't suffer from false positives then you have nothing to worry about because your text won't be detected.

Re: How Claude marks AI-generated content

#445

Post by platinumrad »

dns_snek wrote: post #444
platinumrad wrote: post #442

Earlier quoted context omitted.

If you take it as given that the watermark detector won't have "plausible false positives" then, sure, your argument becomes trivial. I'm not taking that as given.

I'm not taking anything as given. If it suffers from false positives then you have nothing to worry about because the signal won't have any merit -- just like existing AI detectors today . If it doesn't suffer from false positives then you have nothing to worry about because your text won't be detected.

These things aren't binary. There is a large middle ground, which is where Pangram sits today. It's good enough to be useful, but I absolutely would not be willing to fail a student on the basis of a Pangram positive.

If you are implying that this Claude watermark may be good enough to enter the "evidence to fail a student" category then we have a major disagreement.

Re: How Claude marks AI-generated content

#446

Post by focxle »

xp84 wrote: post #408
mchusma wrote: post #230

Many good comments here. It’s somewhat common for me to voice record say a blog post of product updates, more like a ramble. Then have Claude clean it up. Then argue back and forth about certain things until it’s good, then make a final pass sometimes to change a few key words. This is incredibly different than pure ai text. Presumably it will show as ai generated here, even though I would argue it is not really. So…

IDK. At my job it seems like it's expected you'll use it in that way. In academia they've got their own concerns of 'purity,' (not least of which is justifying their continued existence which is in my opinion hard to do) and they are the ones who are going to want most strongly to punish anyone who uses AI. And perhaps "journalists," who will want to trumpet the latest government's press release being [what they'll p…

[dead]
Post reply on HN