Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Claude marks AI-generated content

support.claude.com

Post reply on HN
Print view View original

231–240 of 446 posts

Re: How Claude marks AI-generated content

#231

Post by StilesCrisis »

nemomarx wrote: post #228
wrsh07 wrote: post #215

Earlier quoted context omitted.

I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector) Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doabl…

This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it. I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can ofte…

It's been done and showed up on HN recently. Older content was quite consistently marked as not-AI.

Re: How Claude marks AI-generated content

#232

Post by EE84M3i »

wrsh07 wrote: post #221
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Scott Aaronson talks about his project at OpenAI to do this^ You can carefully select which pseudorandom number generator (prng) you use to be able to id text of a certain length. I expect there is some performance characteristic you have to manage since you're doing this on every inference, but once you do that it doesn't change the output in any meaningful way (the prng is still a statistically valid prng, it just…

https://scottaaronson.blog/?p=6823 is the source?

Re: How Claude marks AI-generated content

#233

Post by andrewgleave »

miohtama wrote: post #86
simonw wrote: post #15

> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response. I'd like to know a lot more about how that works. A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know ex…

Maybe there is a reason why Opus 5 produces such word salad conversations

Yes. The irritating epigram / aphorism style it now uses is such a regression compared to previous Anthropic models. Probably is the case that this is due to watermarking - though hardly subtle if it is.

Re: How Claude marks AI-generated content

#235

Post by wtfwhateven »

sureMan6 wrote: post #127
Dilettante_ wrote: post #120

I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over t…

Maybe it has no false positive rate

Why would you say something so ridiculous?

Re: How Claude marks AI-generated content

#236

Post by bufbupa »

sureMan6 wrote: post #127
Dilettante_ wrote: post #120

I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over t…

Maybe it has no false positive rate

Sorry you're getting downvoted, this interpretation doesn't seem that far fetched to me.

Here's the strawman: The text-based watermarking is going to be done procedurally instead of generatively. Maybe they add some sequence of zero-width Unicode characters to all generated text at certain intervals. Then, there is effectively no false positive possible (because humans would [effectively] never type such sequences of unicode naturally). It may survive some editing (depending on how you select/edit the characters), and it's possible to be stripped (false negatives).

Re: How Claude marks AI-generated content

#237

Post by basch »

wrsh07 wrote: post #214
sureMan6 wrote: post #127

Earlier quoted context omitted.

Maybe it has no false positive rate

Somewhat trivially, if I ask Claude to transcribe an image and then check if that transcription is ai generated it will likely say yes. Many users are not smart enough to realize that the transcription step is where the ai (watermarks) were necessarily injected.

How is a perfect transcription of an image watermarked?

Re: How Claude marks AI-generated content

#238

Post by nunez »

Dilettante_ wrote: post #120

I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over t…

I think Pangram is way ahead of Anthropic on this with their custom dataset.

Re: How Claude marks AI-generated content

#240

Post by nunez »

wrsh07 wrote: post #215
akersten wrote: post #193

Earlier quoted context omitted.

If there is any false positive rate (which, because text will naturally and by chance include tokens from the green and red sets in some pattern, there will be), tools making promises like "detect AI-generated text" are unacceptable. They are going to turn innocent people into pariahs on some unsubstantiated "this content is 37% likely to be AI" claim that the user has no way of verifying or inspecting more deeply, w…

I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector) Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doabl…

pangram is pretty good; i use it all of the time and pay for it. surprised that it's not mentioned that often here. they just released a new model that is supposed to lower the fpr (false positive rate) even further than it's already impossibly low score. it also detects AI in images now, though I expect the fpr to be pretty high there given its newness.
Post reply on HN