Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

31–40 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#31

Post by calebh »

docheinestages wrote: post #24

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing detail…

The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/

For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#32

Post by OneManyNone »

AnthonyMouse wrote: post #30
stymaar wrote: post #11

Earlier quoted context omitted.

> but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do). But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy…

> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. There are two problems with this. The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maxim…

All of that may be true, but pangram currently has a false positive rate of about 1 in 10000, and this has been tested by feeding in thousands of texts written before 2020.

That may not last if AI companies start trying to build models that fool it, but for the time being at least, modern models do have strong tells.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#33

Post by therealdrag0 »

richard_chase wrote: post #29

Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.

I wouldn’t go that far… but it can be kinda like Wikipedia, clean and readable.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#34

Post by visarga »

calebh wrote: post #31
docheinestages wrote: post #24

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing detail…

The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/ For Google Doc users, you can already inspect the edit history over time…

That human might have used AI. You can never know. Hand fixed AI output, human just polished the corners? Light rewording of a full text written by hand, because the author is not confident in their writing? Actual human text, but after researching with AI?

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#35

Post by rayval »

teeray wrote: post #12

The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.

As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.

If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.

EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#37

Post by AnthonyMouse »

OneManyNone wrote: post #32
AnthonyMouse wrote: post #30

Earlier quoted context omitted.

> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. There are two problems with this. The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maxim…

All of that may be true, but pangram currently has a false positive rate of about 1 in 10000, and this has been tested by feeding in thousands of texts written before 2020. That may not last if AI companies start trying to build models that fool it, but for the time being at least, modern models do have strong tells.

You can get an arbitrarily low false positive rate by sacrificing against false negatives. It's trivial to make it zero, just classify everything as human-generated. Meanwhile a false negative rate of even 1% is a pretty big problem since someone can easily use LLMs to generate 100x the volume of text and then use whichever ones make it through the classifier.

And that's before anyone even tries to get the LLM to generate a different style of text. Or for that matter creates a "style model" that rephrases text.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#38

Post by stymaar »

AnthonyMouse wrote: post #30
stymaar wrote: post #11

Earlier quoted context omitted.

> but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do). But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy…

> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. There are two problems with this. The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maxim…

> The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".

The thing is, humans are significantly worse at maximizing numerical goals than computers.

> And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.

Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#39

Post by thatjoeoverthr »

docheinestages wrote: post #24

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing detail…

Sufficiently advanced AI use is probably fine. The slop everyone complains about has certain tells specifically due to some combination of the following:

- The author is conducting some kind of hustle.

- The author doesn't bother editing.

- The author lacks the taste and awareness enough to see it looks.

- The author thinks you, the reader, lack taste and awareness.

- The author is using it as a kind of smoke bomb to get rid of you.

In such cases, nothing is done about the LLM's distinctive "voice". It dominates the text and it's easy to detect. It stands as a signifier of the above, even if it's otherwise not intrinsically a problem to use AI.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#40

Post by vidarh »

OneManyNone wrote: post #32
AnthonyMouse wrote: post #30

Earlier quoted context omitted.

> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. There are two problems with this. The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maxim…

All of that may be true, but pangram currently has a false positive rate of about 1 in 10000, and this has been tested by feeding in thousands of texts written before 2020. That may not last if AI companies start trying to build models that fool it, but for the time being at least, modern models do have strong tells.

Pangram won't know how much AI written text they fail to detect, though, and detectors is a great tool to adjust methods of generating less AI-sounding text.
Post reply on HN