Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

111–120 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#111

Post by vlovich123 »

janalsncm wrote: post #85
yorwba wrote: post #64

Earlier quoted context omitted.

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

You definitely can rule out the general case a priori . If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the…

It’s worse. If the data was separable in this way, you would equally be able to train an AI to mask those signs.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#112

Post by itemize123 »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

obviously, a universal model doesn't exist since the signals are non-stationary but it's way better than what tarot reading

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#113

Post by Ferret7446 »

bob1029 wrote: post #98
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

With sufficient information you can derive a signal even in the presence of overwhelming noise. Assuming the noise is not perfectly correlated with the signal this is always possible. Schemes like GPS, CDMA and DSSS are based upon this concept. GPS in particular is quite impressive in its ability to recover information that is received below the thermal noise floor.

There has to be a signal to detect it.

Take this sentence: Bob went to the store to buy milk.

Was that AI generated or not? There simply isn't a signal there. The problem isn't noise, the problem is, is there even a signal to begin with.

Sure, you might be able to recognize the quirks of a specific LLM just as you recognize the quirks of a particular person, but as the number of LLMs proliferate, then the signal turns into noise. (The signal isn't buried by noise, it becomes noise. The signal no longer has any discriminating power.)

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#114

Post by ekelsen »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

See the discussion on https://news.ycombinator.com/item?id=48837460

You can absolutely still tell.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#115

Post by WhitneyLand »

Der_Einzige wrote: post #109
WhitneyLand wrote: post #71

Earlier quoted context omitted.

"It's really easy to have a false positive" Not really. The false positives for the SOTA detector are very very low. "It's also very easy to change the pattern of LLM output." Not in a way that can reliably avoid detection. The problem is the patterns are baked into the distribution itself. It's smoothed over, so it becomes difficult to prompt your way out of that.

Wrong. Effective sampling (I.e high temperature like temp 10) with the corresponding sampler stack that enables this coherently destroys all attempts to detect it. There are many more ways like this involving manipulating the logprobs

I’m not sure what you’re saying I’m wrong about.

The comment I was responding to, about changing LLM output, referred to prompting, not temp/sampling tricks. I’m not aware of Pangram being beat by clever prompting. There’s some interesting work on creative writing using contrastive prompt techniques, but I haven’t seen it tried as evasion.

Even if you control temp and sampling, they’re not magic. If you raise the temperature too much writing can go to hell, so you may beat the detector but end up with junk. There are some ways to mitigate such a quality drop like raising temperature in conjunction with min-p, but still, I haven’t read any research that shows it getting good results at anything close to 10.

Now you want to get more clever and manipulate logprobs…well ok, you could come up with elaborate strategies designed to evade specific detection methods. But I don’t see that getting done as a weekend project while maintaining writing quality. And if it does happen there’s no guarantee the detector can’t train on its characteristics and start an arms race.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#116

Post by WhitneyLand »

hgoel wrote: post #92
WhitneyLand wrote: post #84

Earlier quoted context omitted.

That’s a different point. I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested. And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process. You need good tools and good oversight.

>And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process. Agreed, that's a fair and reasonable stance. The reason I asked is that I have a hard time understanding the point of these tools. When it comes to education, it can be a matter of learning objectives. But outside that, what's the point? The prediction from the tool is pointles…

I’m personally interested in it as part of the research to improve LLM writing. Detecting “AI voice” is part of understanding what’s wrong with it in the first place and how to improve it.

But yeah, in general I think you’re right, the actual utility is pretty niche.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#117

Post by ben_w »

vlovich123 wrote: post #111
janalsncm wrote: post #85

Earlier quoted context omitted.

You definitely can rule out the general case a priori . If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the…

It’s worse. If the data was separable in this way, you would equally be able to train an AI to mask those signs.

IIRC, the big names in LLMs have no real interest in cloaking the LLM-nature of the text, Google adds deliberate watermarks to text, OpenAI developed a watermark for text but reportedly arent't actually using it.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#118

Post by vlovich123 »

ben_w wrote: post #117
vlovich123 wrote: post #111

Earlier quoted context omitted.

It’s worse. If the data was separable in this way, you would equally be able to train an AI to mask those signs.

IIRC, the big names in LLMs have no real interest in cloaking the LLM-nature of the text, Google adds deliberate watermarks to text, OpenAI developed a watermark for text but reportedly arent't actually using it.

Considering [1], I’m going to challenge that their techniques are currently even mildly effective. Given the absolute academic malpractice these papers are pushing, I’m calling BS; while they want to watermark it, they clearly aren’t actually able to. For images. Which are drastically easier than text.

Their interest is irrelevant in the face of technical impossibility. And that’s before you get into other people who don’t care and will just build adversarial tools to bypass the attempted watermarks. It’s a losing useless battle. Google and OpenAI engage in it to try to catch competitors when there’s a lawsuit or to try to clean their datasets clean.

But it’s absolutely unusable for something like “did someone cheat”.

[1] https://hackerfactor.com/blog/index.php?/categories/1-Image-...

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#119

Post by alok-g »

40four wrote: post #26

I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solv…

>> don’t see how trying to “detect” LLM generated texts is ever going to work

... if the assumption is LLMs are being optimized to evade such detection.

PS: I didn't read the actual article.

I think it is instruction-tuning that is having LLMs write differently from humans, and this is not being optimized away.

Post reply on HN