Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

81–90 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#81

Post by onestay42 »

yorwba wrote: post #64
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

Not all humans are in their right minds, unfortunately.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#82

Post by stymaar »

pixl97 wrote: post #73
stymaar wrote: post #66

Earlier quoted context omitted.

I don't think it's a very good remark, as there's significantly less email spam than 20 years ago. Another example is ad-blocker-blocker. There was a little bit of an arm race between ad blockers and advertisers in the middle of the 2010s, but it didn't last long. Advertisers mostly just decided not to care about ad-blockers.

>Advertisers mostly just decided not to care about ad-blockers. Directly not to care because they lost in court. And yet the biggest advertizer on Earth (Google) decided to change their browser to make adblocking far more difficult. That or they say "just use an app, oh and turn on notifications". I'm not exactly sure who you think won the arms race there, but it seems like we the user did not. There is significantly…

> Directly not to care because they lost in court.

I'm pretty sure the illegal sport streaming websites didn't stop doing that just because it became illegal, otherwise they could have stopped their activity altogether while they were at it…

> I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.

I, at least, won when the webiste showing ads gave up the race (for the past decade at least, only time will tell about the future).

> nd yet the biggest advertizer on Earth (Google) decided

This is actually an argument in my direction! The owners of websites (which are also the ones posting slop today) didn't care enough and the situation only changed because Google moved.

I expect the same thing with slop. Individual websites won't make any effort to make their slop unblockable, and it will only be a problem if OpenAI/Anthropic/Google decide that they care about this market. But unlike Google in the ads market, I don't think the model providers have any reason to care. The web is already dead in their mind anyway.

> There is significantly more spam than 20 years ago, just less of it reaches your inbox.

This goes against your very argument from earlier!

> On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them.

Out of convenience, but you don't need that to be practically free of spam. Whatever version of SpamAssassin is being run on OVH's mail servers has been enough for that purpose for me.

> they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.

Again, they don't care about the web. They just crawl it for content but they don't want you to read any webpage, they want you to stay in their chatbot. Every other use-case is nonexistent to them (except coding agents, of course, but that's a different product altogether).

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#83

Post by vidarh »

AnthonyMouse wrote: post #51
vidarh wrote: post #42

Earlier quoted context omitted.

You don't really need a style model - current models are very good at doing "style transfer" of a model text onto whatever it has written if you just have it do it chunk by chunk. It takes more to prevent it from being detectable by good detectors, but it does remove a lot of the worst tells.

The point being that you wouldn't need the developers of the most popular models to themselves be trying to fool classifiers because their output could be run through an independent special purpose one designed to remove the tells the classifier is looking for, and the special purpose one wouldn't need to be made by anyone with the resources to create a good general-purpose model since it only has to do that one thin…

My point is that you don't need a special purpose one to achieve this.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#84

Post by WhitneyLand »

hgoel wrote: post #80
WhitneyLand wrote: post #67

Earlier quoted context omitted.

"Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it...it's a bad fiction to perpetuate that any of this is anything more than tarot card reading." Not true at all. Pangram is highly effective and has a very low false positive rate. The post here is impressive for a small project, it looks like they independently thought of one of the core ideas Pangram uses of…

So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work? If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?

That’s a different point.

I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested.

And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process.

You need good tools and good oversight.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#85

Post by janalsncm »

yorwba wrote: post #64
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

You definitely can rule out the general case a priori. If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible.

As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the labels conflict for the same piece of text.

The data is statistically inseparable which makes it impossible to classify from text alone.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#86

Post by cwmoore »

onestay42 wrote: post #81
yorwba wrote: post #64

Earlier quoted context omitted.

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

Not all humans are in their right minds, unfortunately.

It is much harder to tell one from the other, and for oneself, than it often seems on the surface.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#87

Post by Wowfunhappy »

mr_mitm wrote: post #75
Wowfunhappy wrote: post #63

Earlier quoted context omitted.

…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated. If I can do it, an algorithm should be able to do it. Maybe in the future the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.

How do you estimate your false negative rate?

No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair.

Edit: But I'm also super conflicted about this, because I really want to read what humans think, not what an AI thinks, regardless of the writing quality.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#88

Post by cwmoore »

WhitneyLand wrote: post #84
hgoel wrote: post #80

Earlier quoted context omitted.

So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work? If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?

That’s a different point. I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested. And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process. You need good tools and good oversight.

Due process should never just become a checkbox item. To deal with lives and livelihoods justly, you need appeal pathways and meaningful liability exposure for the processors.

Plagiarism and cheating sucks for everyone. Worth solving.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#89

Post by throaway54321 »

I don’t think it actually matters (and it’s a losing strategy as others have noted).

This issue with AI generated stuff is that that it’s sometimes asymmetric: either the author worked very little to produce a lot of slop and now the reader(s) all have to do the heavy effort of reading it OR the author puts a little extra work in once and resolves all future readers’ burden.

If it was possible to boil down an artifact into a prompt + some resources that would be an interesting tool, or at least some way to tell if some artifact is “worth my time to read”

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#90

Post by sigmoid10 »

yorwba wrote: post #64
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

That's like saying whether or not you're going to fall in love this year is just one bit of information, so you might be able to read it from astrology. Yeah, sure, it might happen for some people with a certain star sign. But across the population there is zero reason to believe that there is a) any significant correlation and b) enough data variation in to even distinguish classes of humans.
Post reply on HN