Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

131–140 of 446 posts

Re: How Claude marks AI-generated content

#131

Post by platinumrad »

colesantiago wrote: post #130
platinumrad wrote: post #122

Earlier quoted context omitted.

Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.

Then what is the best way to know that something is AI generated slop then?

The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway.

Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up.

You might have to make peace with the fact that there may not always be a tool that does what you want.

Re: How Claude marks AI-generated content

#132

Post by colesantiago »

pixl97 wrote: post #69
colesantiago wrote: post #65

Earlier quoted context omitted.

(This is the part where you provide extensive extraordinary evidence to your claim)

No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.

You're saying because it has some false positives that Pangram is 100% a scam?

Is their research also a scam too?

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/pangram-4-technical

If so, what is the best one out there other than Pangram then?

Re: How Claude marks AI-generated content

#133

Post by colesantiago »

rcxdude wrote: post #113
colesantiago wrote: post #65

Earlier quoted context omitted.

(This is the part where you provide extensive extraordinary evidence to your claim)

Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

This is on Pangram 3 which is very very old now and the founder responded below

https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

What about on Pangram 4?

https://www.pangram.com/blog/pangram-4-technical

Re: How Claude marks AI-generated content

#134

Post by iamherrylok »

akozak wrote: post #30
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Most likely this method https://arxiv.org/pdf/2301.10226 (EDIT: and Google's SynthID paper which builds on it https://www.nature.com/articles/s41586-024-08025-4 )

If different model providers use different green logits, does that mean they can only tell if the text came from their own model?

Re: How Claude marks AI-generated content

#135

Post by colesantiago »

platinumrad wrote: post #131
colesantiago wrote: post #130

Earlier quoted context omitted.

Then what is the best way to know that something is AI generated slop then?

The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway. Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up. You might have to make peace with the fact that there may not always be a tool that does what you want.

So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger.

Thanks!

> But the burden of proof is on them...

I mean is this enough proof?

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

Or is this marketing, a public stunt or not real research?

I think this is enough for me to know they are actually improving their AI slop detector.

Re: How Claude marks AI-generated content

#136

Post by timpera »

shinryuu wrote: post #102
sixtyj wrote: post #88

Earlier quoted context omitted.

It was quick :) … https://claudewatermarkremover.app/

Though if pangram should be trusted, there are still statistical artifacts that tells you that a text LLM generated. I don't find that to be implausible.

Alas, Pangram should not be trusted.

Re: How Claude marks AI-generated content

#137

Post by ebtebt »

COAGULOPATH wrote: post #80
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…

Just double checking my understanding: If this is true then only Anthropic will be able to detect if text was generated by one of its models, correct?

Re: How Claude marks AI-generated content

#138

Post by JohnKemeny »

jonplackett wrote: post #101

We need to just stop pretending we can reliably tell if plain text is written by an LLM. It’s just not a reasonable ask.

True, but what you can do is a one-sided guarantee. If it bears the mark, it is likely generated (or someone deliberately made it look generated).

Thus, if a news article, research article, book, student paper submission, blog post, HN comment, etc, bears the mark, it could be automatically flagged as such.

It helps detect low effort slop.

---

Caveat. If you write your own creative work and send it to Claude for "cleaning up grammar", it might insert the watermark.

Re: How Claude marks AI-generated content

#139

Post by infinite_spin »

simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

My guess is it will be similar to how Genius watermarked lyrics, using things like variants of punctuation

https://www.pcmag.com/news/genius-we-caught-google-red-hande...

Re: How Claude marks AI-generated content

#140

Post by ForHackernews »

ebtebt wrote: post #137
COAGULOPATH wrote: post #80

Earlier quoted context omitted.

>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…

Just double checking my understanding: If this is true then only Anthropic will be able to detect if text was generated by one of its models, correct?

Likely yes.
Post reply on HN