Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#131

Post by lemagedurage »

janalsncm wrote: post #85
yorwba wrote: post #64

Earlier quoted context omitted.

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

You definitely can rule out the general case a priori . If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the…

That just proves that perfect classification is impossible. Classification doesn't need to be 100% accurate to be useful.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#132

Post by mschild »

wzdd wrote: post #128
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

The article discusses a technique by which the author achieves high accuracy at detecting AI written text. Unless you have a problem with their experimental method, this is the opposite of tarot card reading. > we are well into undetectable sophistication with today's models The article directly contradicts this, as do you, in your previous paragraph: "Sure you might be able to detect today's tells". The article is l…

It does achieve high accuracy but I think given the context when one wants to know this information, plagarism for research papers and college/highschool essays and work, it's unfortunately not good enough.

My neighbour is a teacher. She has a really good idea which of her students uses AI to do their homework but 80% accuracy is not good enough. She'd need to be able to prove it with certainty.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#134

Post by AnthonyMouse »

stymaar wrote: post #60
AnthonyMouse wrote: post #44

Earlier quoted context omitted.

> The thing is, humans are significantly worse at maximizing numerical goals than computers. I'm not sure this is even the right premise. Existing LLMs try to maximize engagement, and they often write in a particular style that has tells, but these two things are not necessarily related. Over-using em-dash or whatever isn't the thing that maximizes engagement. So the two problems really are, what happens to the actua…

> Over-using em-dash or whatever isn't the thing that maximizes engagement. It's the thing that minimizes the loss during the RLHF phase, and the RLHF phase is the one that's aimed at maximizing engagement (it's literally trained on that). > what happens to the actual humans whose writing style is a close match for what a given generation of LLMs output? If a human, for instance because its writing gets polluted by r…

> It's the thing that minimizes the loss during the RLHF phase, and the RLHF phase is the one that's aimed at maximizing engagement (it's literally trained on that).

I don't think RLHF is the biggest reason its style is the way it is.

A lot of it is that it's trained on everything they could get their hands on, which includes domain-specific literature and books that go all the way back to the advent of writing, and then will pick up habits that are common in some specific domain or in 19th century literature etc. that are less common in most modern writing when no attempt is being made to do otherwise.

Do you really think that RLHF humans were requesting more em-dash?

> Websites have pretty much stopped using ad-blocker-blockers, it seems that it's not a fight worth fighting for them. Does that mean that ad-blockers are useless?

Websites have pretty much stopped using them because they realized readers with ad blockers will stop using the site sooner than stop using their ad blocker, and since websites have a network effect, it's better to let a minority of readers block ads when having them makes it more likely they'll distribute links to the site. And because it's the user who controls the browser for web pages, which gives ad blockers a decisive advantage.

> Most people don't even care about ads, I don't think they care about slop either, that's why there's slop posts and obnoxious websites that are unreadable without an ad blocker. A slop blocker used by 10-20% of the internet users wouldn't change the calculation more than ad blockers did.

Sites don't want users to use ad blockers, but having a user with an ad blocker is still better for them than not having the user at all, because of the network effect.

Whereas many sites don't want slop at all, and then if slop detectors work they'll put them in the site itself and block the slop for 100% of users. At which point the slop generators have a 100% incentive to find a workaround instead of a 20% incentive, which is different.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#135

Post by lelanthran »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.

This is simply untrue, and completely divorced from reality.

Tarot card readings have literally zero predictive success. Last I checked, LLM-detection had a +90% success.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#136

Post by sigmoid10 »

lemagedurage wrote: post #130
sigmoid10 wrote: post #90

Earlier quoted context omitted.

That's like saying whether or not you're going to fall in love this year is just one bit of information, so you might be able to read it from astrology. Yeah, sure, it might happen for some people with a certain star sign. But across the population there is zero reason to believe that there is a) any significant correlation and b) enough data variation in to even distinguish classes of humans.

The 80% accuracy from the article would be one reason to believe there's significant correlation, no?

I could probably find quite a lot of people who will tell you astrology is 80+% correct for them. Would you believe them or wait for an independent analysis? There are other AI "detector" systems out there that claim 99% accuracy. But independent research always found that they are actually garbage once used on real data. It's all in how you pick your tests. It's also funny to see how people on places like HN will easily dismiss stuff astrology, but fall for the exact same patterns when used in tech-y applications.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#137

Post by lelanthran »

hgoel wrote: post #80
WhitneyLand wrote: post #67

Earlier quoted context omitted.

"Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it...it's a bad fiction to perpetuate that any of this is anything more than tarot card reading." Not true at all. Pangram is highly effective and has a very low false positive rate. The post here is impressive for a small project, it looks like they independently thought of one of the core ideas Pangram uses of…

So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work? If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?

>>> "Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it...it's a bad fiction to perpetuate that any of this is anything more than tarot card reading."

>> Not true at all. Pangram is highly effective and has a very low false positive rate.

> So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work?

What point are you arguing? Something having a high success rate does not necessarily translate to treating it as a 100% success rate.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#138

Post by SiempreViernes »

mschild wrote: post #132
wzdd wrote: post #128

Earlier quoted context omitted.

The article discusses a technique by which the author achieves high accuracy at detecting AI written text. Unless you have a problem with their experimental method, this is the opposite of tarot card reading. > we are well into undetectable sophistication with today's models The article directly contradicts this, as do you, in your previous paragraph: "Sure you might be able to detect today's tells". The article is l…

It does achieve high accuracy but I think given the context when one wants to know this information, plagarism for research papers and college/highschool essays and work, it's unfortunately not good enough. My neighbour is a teacher. She has a really good idea which of her students uses AI to do their homework but 80% accuracy is not good enough. She'd need to be able to prove it with certainty.

not really, a strong suspicion is enough to motivate assigning an extra paper and pen in person test to a student, and then you can fail them on that result.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#139

Post by SiempreViernes »

janalsncm wrote: post #85
yorwba wrote: post #64

Earlier quoted context omitted.

Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that. As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent…

You definitely can rule out the general case a priori . If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible. As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the…

> But since humans and machines have both written many texts, it is not possible.

Maybe you meant "many humans have used AI when writing texts"? Your stated reason that they can't be separated because there are many texts of each kind is nonsensical, you clearly need to supply more reasons than "there are many".

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#140

Post by SiempreViernes »

lemagedurage wrote: post #129
seanmcdirmid wrote: post #127

Earlier quoted context omitted.

If you have access to the detector, you can formulate a generative solution that avoids being flagged. Which gets me wondering why don’t model providers do that? There must be something about that that destroys semantic weights somehow.

Why would sounding human be a goal rather than a byproduct of trying to communicate efficiently?

One of the big use case is cheating on your essay assignment.
Post reply on HN