Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

151–160 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#151

Post by dofm »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it.

This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated.

It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sounds like an averaged Californian college student to my British grammar-school-educated ears, as so many of the situations where I am encountering AI text are Brits using it without apparently realising they are giving themselves away.

But I know people who don't have particular technical skills in this sphere or a grammar-school background who also have an uncanny knack for pointing out LLM-written text.

> Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.

I especially don't think this is true. Will they be able to do it in the future? Maybe. Is it possible to prompt a current cloud LLM to write in a way that is obvious? Yeah. (IMO Gemma 4 writes less detectably than most of them!)

But my instinct is that someone with any facility for language is going to be better than chance at spotting LLM-written text once it is three or four paragraphs long. So I think it should be possible in principle to train machine learning systems to detect those patterns.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#152

Post by dofm »

TacticalCoder wrote: post #94
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

> ... but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading Most people's issue with AI-generated llmish however is not that it's AI-generated. It's its insufferable tone. So if we get to a point where we have to read tea leaves (an image you seem to appreciate) to determine if it's llmish or not, we'll have won by then. Really: it's that full-on asshole tone I (and many other…

My broad feeling is that if we generally independently identify the insufferable tone, and we absolutely can, so can a sufficiently trained machine-learning model.

It's an aside, but my biggest problem with trying to get up to date and learn about LLMs is how much of the documentation, blog writing, and tutorial material has obviously been written by no-one. It is just so much harder to read (and, like generative AI slop generally, curiously much harder to recall later).

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#153

Post by CalRobert »

dofm wrote: post #151
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated. It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sou…

If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them?

I do struggle at times with thinking my own writing looks like AI. But I’m an average Californian who went to college half way between SF and LA…

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#154

Post by CalRobert »

SiempreViernes wrote: post #138
mschild wrote: post #132

Earlier quoted context omitted.

It does achieve high accuracy but I think given the context when one wants to know this information, plagarism for research papers and college/highschool essays and work, it's unfortunately not good enough. My neighbour is a teacher. She has a really good idea which of her students uses AI to do their homework but 80% accuracy is not good enough. She'd need to be able to prove it with certainty.

not really, a strong suspicion is enough to motivate assigning an extra paper and pen in person test to a student, and then you can fail them on that result.

That seems pretty unfair. Why not make the original test pen and paper then? (Or at least a typewriter, offline computer, etc - my handwriting is awful)

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#155

Post by dofm »

CalRobert wrote: post #153
dofm wrote: post #151

Earlier quoted context omitted.

> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated. It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sou…

If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them? I do struggle at times with thinking my own writing looks like AI. But I’m an average Californian who went to college half way between SF and LA…

> If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them?

I don't know. I mean, it feels like the systems that would detect them are likely qualitatively different to the machines that make them.

One of the things that feels obvious to me is that LLMs are always going to write in a new way, because words do not get all that close to perfectly conveying the inner thoughts of competent writers. Competent writing is always a battle to find the better word, or even to create it.

So sure, you could add another adversary that the generator has to satisfy, but "this sounds like a machine wrote it" is only an observation; it's not a prescription for not writing like a machine.

Maybe it's never going to be possible.

> I do struggle at times with thinking my own writing looks like AI. But I’m an average Californian who went to college half way between SF and LA…

:-)

You guys do just sound a certain way, in the same way Brits sound a certain way to you I expect. But I think the reality is that the final stage of training LLMs was largely done in a Californian voice and with rather Californian communication objectives.

(Though equally I think much of what I am detecting is more Madison Avenue than Palo Alto)

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#156

Post by [deleted] »

CalRobert wrote: post #153
dofm wrote: post #151

Earlier quoted context omitted.

> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated. It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sou…

If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them? I do struggle at times with thinking my own writing looks like AI. But I’m an average Californian who went to college half way between SF and LA…

[deleted]

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#157

Post by CalRobert »

dofm wrote: post #155
CalRobert wrote: post #153

Earlier quoted context omitted.

If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them? I do struggle at times with thinking my own writing looks like AI. But I’m an average Californian who went to college half way between SF and LA…

> If you can train a system to detect these patterns, presumably you can train systems not to generate text which matches them? I don't know. I mean, it feels like the systems that would detect them are likely qualitatively different to the machines that make them. One of the things that feels obvious to me is that LLMs are always going to write in a new way, because words do not get all that close to perfectly conve…

Perhaps mistral will save us all from sounding like Californians. But sure it’s grand, you know yourself :-)

(Haven’t lived in California in a long time now)

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#158

Post by mr_mitm »

Wowfunhappy wrote: post #87
mr_mitm wrote: post #75

Earlier quoted context omitted.

How do you estimate your false negative rate?

No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair. Edit: But I'm also super conflicted about this, because I really want to read what humans think, not what an AI thinks, regardless of the writing quality.

My point is that you cannot know that AI generated text is obviously AI generated. It's like the old "hair pieces look awful, I can tell immediately!" - no, you can only tell when they're awful. Maybe you're not as good as you think at detecting AI.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#159

Post by Tade0 »

Ferret7446 wrote: post #113
bob1029 wrote: post #98

Earlier quoted context omitted.

With sufficient information you can derive a signal even in the presence of overwhelming noise. Assuming the noise is not perfectly correlated with the signal this is always possible. Schemes like GPS, CDMA and DSSS are based upon this concept. GPS in particular is quite impressive in its ability to recover information that is received below the thermal noise floor.

There has to be a signal to detect it. Take this sentence: Bob went to the store to buy milk. Was that AI generated or not? There simply isn't a signal there. The problem isn't noise, the problem is, is there even a signal to begin with. Sure, you might be able to recognize the quirks of a specific LLM just as you recognize the quirks of a particular person, but as the number of LLMs proliferate, then the signal turn…

The article itself explained that it was much easier to classify text as human or LLM generated than to have "human" as just a category along with all the different LLMs as it's likely the LLMs are distilled from each other, creating a unique footprint.

If a signal is weak, it might not even appear in every sentence, but that doesn't mean it doesn't exist. For instance, I don't recall ever consciously using an em dash, but you'll probably need an entire paragraph to find one in LLM-generated text.

My own sense of whether text is generated is partially based on its sheer length - humans typically don't bother writing so much.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#160

Post by Anoian »

cygn wrote: post #105
Krssst wrote: post #4

The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ? If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads. (the article was a good read, thanks!)

I built a browser extension that does this, well for posts on twitter, hackernews, reddit etc. If you want it for all text, it would also be feasible. I use a quantized mini-LM model that runs very fast and classifies eg your whole twitter feed in a couple of seconds. Check it out: https://slopsieve.com/extension Accuracy is also much higher than this approach here. 0.9944 AUC, 0.966 acc@.5, 0.971 F1@.5

Went to the website and inserted the blogposts I wrote in the last year. I had a pretty good understanding which of my blogposts had more reworks and which ones had entire passages being generated using AI and then left as is because I was happy with them. None of my articles were over 50% according to your model but the ones that I know took me a long time to write, even though I used lots of AI in the creation of them, hit below 10%, probably because I hand-edited them a lot. Overall a nice website, thanks for sharing :)
Post reply on HN