will this solve anything? people will build tools to remove the watermarks
How Claude marks AI-generated content
261–270 of 446 posts
Re: How Claude marks AI-generated content
#262[dead]
Re: How Claude marks AI-generated content
#263Is the detection mechanism going to be open, free, and possible to run locally without prostrating to an opaque third-party company that will do whatever they want with the text content provided (including using it for training), and take no responsibility in case of false-positives for which there can exist no proof or evidence against by the victim? This is another useless, if not actively harmful, performative EU…
It is of course a stupid regulation, but the upside is that it will probably accelerate growth in usage of open models that are not adversarial towards the user.
Re: How Claude marks AI-generated content
#264Earlier quoted context omitted.
If there is any false positive rate (which, because text will naturally and by chance include tokens from the green and red sets in some pattern, there will be), tools making promises like "detect AI-generated text" are unacceptable. They are going to turn innocent people into pariahs on some unsubstantiated "this content is 37% likely to be AI" claim that the user has no way of verifying or inspecting more deeply, w…
I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector) Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doabl…
Pangram has their FP rate and FN rate rates here:
https://www.pangram.com/research/model-card/pangram-4
> Pangram 4 achieves a 0.0041% false positive rate (roughly 1 in 24,000) on 1,000,000 human-written English FineWeb evaluation examples
> Overall False Negative Rate is 0.3396% on English AI generations (26 generator models)
Re: How Claude marks AI-generated content
#265Earlier quoted context omitted.
If there is any false positive rate (which, because text will naturally and by chance include tokens from the green and red sets in some pattern, there will be), tools making promises like "detect AI-generated text" are unacceptable. They are going to turn innocent people into pariahs on some unsubstantiated "this content is 37% likely to be AI" claim that the user has no way of verifying or inspecting more deeply, w…
I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector) Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doabl…
Language distribution shifts. Eventually people will start adopting the distribution used by LLMs, making classification harder.
Also, this doesn't even consider the case where people use LLMs to translate their original works. Or people that use it for spelling/grammar checks.
Personally, I believe these checkers do more harm than good. Any false positive can ruin someones life.
Re: How Claude marks AI-generated content
#266The more they fiddle with the autocomplete system, the more they move away from the autocomplete faithfully producing the completion I need. The more it makes sense to move to an open weights model not served by them.
Re: How Claude marks AI-generated content
#267Earlier quoted context omitted.
Maybe it has no false positive rate
If LLM training data is human-written, and LLM output mimics that input, how could you not have false positives?
Because it won't be in the training directly. It is applied after a model generates its distribution of likely tokens, biasing each token randomly based on a random key and unrelated to any meaning of the words. So half the time, the most likely token becomes more likely and half the time it becomes less likely, and the same for every other token (when temperature is above 0).
You then look at the tokens actually picked to see how closely they follow this pattern that isn't connected to the meaning of the tokens. With enough text, you can then analyze the chance of it happening by chance verses being because the generation of the tokens was done using the algorithm, and you can save a positive result until you are arbitrarily sure. There is a chance of a false positive, but the chance of a false positive approaches the chance that the murderer happened to have fingerprints that matched your and both forensics labs happened to have mixed up the dna tests and the eye witness happened to misremember the face and your phone gps happened to glitch out and put you at the murder scene at the time of the crime all happening. It is theoretically possible only in the same sense that quantum teleporting a cat is theoretically possible.
The real question is how much text do they need for a given level of certainty and what do they check for. If they flag a positive at a p value <.01, that's a problem. If they can reasonably get a p value of < 1e-12 in only a few paragraphs of text, that is effectively no false positives (but a lot of 'too short to analyze' outcomes).
Re: How Claude marks AI-generated content
#268Would this impact distillation? Reminds me of the fake roads map makers would add to their maps to detect copying.
Re: How Claude marks AI-generated content
#269Re: How Claude marks AI-generated content
#270Earlier quoted context omitted.
Less probable also means less optimal and you get a subpar response. More so if it's baked into its reasoning. It's intelligence will suffer unless this is some post processing thing.
There is already some intentional randomness in token selection, because it actually improves the quality of responses if you intentionally don't always pick the most likely next token. You can hide data in that randomness without impacting the quality of the response by using a sufficiently "random looking" pseudorandom bit stream instead of real random numbers. I previously worked on a project to do that here: http…
Good point, very interesting. Thanks!