Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

181–190 of 446 posts

Re: How Claude marks AI-generated content

#181

Post by yedidmh »

baq wrote: post #29
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

> I'd like to know a lot more about how that works. Count load-bearing words using two different algorithms in a belt-and-braces fashion

I have to push back: you've found the seam.

Re: How Claude marks AI-generated content

#182

Post by WiSaGaN »

Dilettante_ wrote: post #120

I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over t…

My guess is that they will later "reveal" some "violations" but provide little evidence citing proprietary algorithm.

Re: How Claude marks AI-generated content

#183

Post by olmo23 »

embedding-shape wrote: post #158
wpietri wrote: post #156

Earlier quoted context omitted.

As anybody who has put together a coding standard knows, there are a lot of options for individual expression, meaning a lot of room for things like watermarking. And of course you can add arbitrary comments; my Claude-generated code is very verbose.

> there are a lot of options for individual expression, meaning a lot of room for things like watermarking The way I use LLMs (and I'd advice everyone to do the same) there really isn't, the agent implements things exactly how I want them, or I use the agent to massage it into the exact bit-by-bit version I imagined when I first sent the prompt afterwards. I honestly don't know what the point would be to let the agen…

> I honestly don't know what the point would be to let the agents compose worse code than what I'd do manually

You never generate throwaway code used to test an external service? or try out an interface idea? There's a lot of code that's only meant to be ran once. I often dont even care what language it's written in.

Re: How Claude marks AI-generated content

#184

Post by embedding-shape »

olmo23 wrote: post #183
embedding-shape wrote: post #158

Earlier quoted context omitted.

> there are a lot of options for individual expression, meaning a lot of room for things like watermarking The way I use LLMs (and I'd advice everyone to do the same) there really isn't, the agent implements things exactly how I want them, or I use the agent to massage it into the exact bit-by-bit version I imagined when I first sent the prompt afterwards. I honestly don't know what the point would be to let the agen…

> I honestly don't know what the point would be to let the agents compose worse code than what I'd do manually You never generate throwaway code used to test an external service? or try out an interface idea? There's a lot of code that's only meant to be ran once. I often dont even care what language it's written in.

> You never generate throwaway code used to test an external service? or try out an interface idea?

And save/persist it? No, most of any experimental stuff goes into /tmp which gets cleared out on reboot, nothing I care to save in any repository. Or just "show me how this would look like" and then it's only in the session itself (and the logs/state I suppose, technically...).

Re: How Claude marks AI-generated content

#185

Post by cubefox »

pixl97 wrote: post #64
colesantiago wrote: post #44

Good. They should make it easier, to detect slop so we can ignore it quickly. I hope Pangram makes an API or an extension to analyze a page to detect slop on a page and then closes the tab immediately. Nobody should be wasting time on garbage LLM output in code, text, image and videos.

Panagram is a scam.

It's not. Pangram is quite accurate. Not being perfect doesn't make it a scam.

Re: How Claude marks AI-generated content

#186

Post by mechanicum »

colesantiago wrote: post #130
platinumrad wrote: post #122

Earlier quoted context omitted.

Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.

Then what is the best way to know that something is AI generated slop then?

Talking to the person who gave it to you, in my experience.

In my own testing, Pangram is excellent at detecting the default output styles of LLMs.

If you tell the LLM to change its output style, so it’s not full of “load-bearing spaced em dashes that aren’t X, they aren’t Y. they’re Z.” constructions (which humans are pretty good at detecting on their own), the false negative rate soars.

Re: How Claude marks AI-generated content

#187

Post by olmo23 »

cassianoleal wrote: post #54
benrow wrote: post #36

I've heard that this kind of watermarking process works by biassing the statistical sampling towards a partition of the set of possible next tokens (red set and green set), at each position. It might only be a slight nudge each time, but over a sequence of tokens, the likelihood of repeating the bias by chance is increasingly improbable. The bias is different for each position and follows a defined RNG, seeded someho…

> a defined RNG, seeded somehow predictably So, an NG?

PRNG

Re: How Claude marks AI-generated content

#188

Post by dannyw »

wpietri wrote: post #160
DaSHacka wrote: post #123

Earlier quoted context omitted.

Thankfully, there are a variety of Chinese models that never will. I think we all know that in a few years, they will also be the only relevant offerings on the market, due to not being bogged down with over-zealous ""safety"" footguns.

Your theory is that the Chinese government is thoroughly uninterested in safety or prosocial controls?

Amongst Chinese labs and netizens, there's MUCH less belief/mindshare on "AGI = existential risk to humanity", "paperclip maximiser", and similar lines of thinking. AI is seen more as just a technology, and less like a scary boogyman.

Whether that's right or wrong, I'll leave to you, but there's huge differences in perspectives, and if you only get your news from Western sources and communities (and companies), you're in a bubble too. A different bubble, and arguably a more porous one, but still a bubble.

Re: How Claude marks AI-generated content

#189

Post by mhjkl »

Mashimo wrote: post #112
hbn wrote: post #72

If the western AI companies are forced to comply with this type of BS, and develop their models to do their job while balancing a book on their head and hopping on one foot, the Chinese models just got a free pass to completely dominate the frontier. EU regulation does it again!

If the Chinese want to sell to EU customers, they probably have to do the same.

I’ve seen Chinese open weights models say “can’t use this if you’re in Europe” in their licenses, so I doubt they would invest too much into complying

Re: How Claude marks AI-generated content

#190

Post by shawnz »

KoolKat23 wrote: post #180
COAGULOPATH wrote: post #80

Earlier quoted context omitted.

>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…

Less probable also means less optimal and you get a subpar response. More so if it's baked into its reasoning. It's intelligence will suffer unless this is some post processing thing.

There is already some intentional randomness in token selection, because it actually improves the quality of responses if you intentionally don't always pick the most likely next token.

You can hide data in that randomness without impacting the quality of the response by using a sufficiently "random looking" pseudorandom bit stream instead of real random numbers.

I previously worked on a project to do that here: https://github.com/shawnz/textcoder

Post reply on HN