Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

171–180 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#171

Post by strgcmc »

lemagedurage wrote: post #144
sigmoid10 wrote: post #136

Earlier quoted context omitted.

I could probably find quite a lot of people who will tell you astrology is 80+% correct for them. Would you believe them or wait for an independent analysis? There are other AI "detector" systems out there that claim 99% accuracy. But independent research always found that they are actually garbage once used on real data. It's all in how you pick your tests. It's also funny to see how people on places like HN will ea…

Well I don't think the position of planets when you're born has a large correlation to how your life will go. On the other hand, how an AI writes will have a big correlation to whether the written text would likely be written by an AI. The latter is more of a direct relationship. Maybe A and B are not correlated, and Y and Z are? What pattern are people falling for here?

It's not really about the planets. It's about what other people believe about the planets.

You could work for a boss that's a Leo, and he/she believes only Leos deserve to get promoted, or that Leos and Scorpios should never be assigned together on a shared project. Your life and career trajectory under this boss could be totally different, depending on whether you were born a Leo or not.

Certainly "not all bosses" applies, but it's not really that farfetched or uncommon either. The point is that a correlation does exist, but it's a social one and not a physical or astronomical one (and it's also often a self-reinforcing/self-fulfilling one: Leos who read what astrology says Leos should do, may end up choosing to behave more like that).

So in the LLM example, it may not really matter much what physical markers of provenance or physical correlations there are, as social beliefs or perceptions about suspected provenance may be the strongest correlation anyways (in terms of impact and outcomes).

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#172

Post by poisonfountain »

My theory is that the labs are RL'ing models to output easily classifiable text so they can avoid model collapse when training the models on data scraped from the web.

Of course, you can skew the distribution with some effort and generate text that avoids even the best classifiers out there (like Pangram), but even tech-savvy people aren't usually doing it (see the amount of AI-written posts that end up in HN and get tons of comments complaining about AI mannerisms), so I guess they're successfully avoiding like 99% of the slop using such classifiers.

I don't think it's in the interest of the labs to allow you to generate text that's indistinguishable from human prose. Especially since nobody would pay $1,000/mo just to generate text - but would do so for tasks like coding.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#174

Post by projektfu »

CalRobert wrote: post #154
SiempreViernes wrote: post #138

Earlier quoted context omitted.

not really, a strong suspicion is enough to motivate assigning an extra paper and pen in person test to a student, and then you can fail them on that result.

That seems pretty unfair. Why not make the original test pen and paper then? (Or at least a typewriter, offline computer, etc - my handwriting is awful)

Originally the idea was that a take-home essay would allow students to work at their own pace, study their own way, and produce something interesting. But if more than half will just prompt an AI and learn nothing, then I agree you should proctor all assessments.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#176

Post by lcnhk42 »

cygn wrote: post #105
Krssst wrote: post #4

The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ? If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads. (the article was a good read, thanks!)

I built a browser extension that does this, well for posts on twitter, hackernews, reddit etc. If you want it for all text, it would also be feasible. I use a quantized mini-LM model that runs very fast and classifies eg your whole twitter feed in a couple of seconds. Check it out: https://slopsieve.com/extension Accuracy is also much higher than this approach here. 0.9944 AUC, 0.966 acc@.5, 0.971 F1@.5

Hey this is really cool. Do you mind sharing a little bit about your approach? Is it similar SVM like algorithm or something else entirely?

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#177

Post by IshKebab »

40four wrote: post #168
IshKebab wrote: post #59

Earlier quoted context omitted.

> I just don’t see how trying to “detect” LLM generated texts is ever going to work He literally demonstrated a working system in this post. Do you mean you'll never get to 100% accuracy? Clearly, but you don't need that.

I just mean no matter how hard anyone tries, I don’t see how useful these systems would be in practice. Sure they demonstrated a “working” system. Plenty companies sell products that “work” to one extent or another. But how useful is it really to get a result of “This is 80% likely chance of being LLM generated”? Or 75%, or 95%? What if the text is a mix of human written text and LLM text? How would you even begin to…

Depends on the application. I would love to have that percentage next to HN submissions so I don't waste time reading (or starting to read) obvious slop. Doesn't really matter if I occasionally skip something that isn't actually slop.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#178

Post by cwmoore »

dofm wrote: post #166
saithound wrote: post #165

Earlier quoted context omitted.

> This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated. I suspect the difficulty here lies more with your reading of the quoted sentence. British grammar school education, for all the years it devotes to the enterprise, does not always succeed in teaching reading comprehe…

> I suspect the difficulty here lies more with your reading of the quoted sentence. British grammar school education, for all the years it devotes to the enterprise, does not always succeed in teaching reading comprehension. Well aren't you a genuine delight? > Alas, even in your own example, when your British friends are "giving themselves away", you resort to an external signal, beyond the text, to determine proven…

Erstwhile elsewhere whilst among the midst of the unbeknownst, someone is singing along to Taylor Swift songs I would not recognize. In a democracy of ideas, it is the lightning and not the cloud that makes it thunder. You Britons do sound smart.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#179

Post by vlovich123 »

ben_w wrote: post #124
vlovich123 wrote: post #118

Earlier quoted context omitted.

Considering [1], I’m going to challenge that their techniques are currently even mildly effective. Given the absolute academic malpractice these papers are pushing, I’m calling BS; while they want to watermark it, they clearly aren’t actually able to. For images. Which are drastically easier than text. Their interest is irrelevant in the face of technical impossibility. And that’s before you get into other people who…

> Their interest is irrelevant in the face of technical impossibility. I'm responding to "If the data was separable in this way, you would equally be able to train an AI to mask those signs.": yes, if you wanted to you could, the big names clearly don't consider masking to be a priority. > But it’s absolutely unusable for something like “did someone cheat”. This is the one case where I'd most expect it to succeed: I…

You’re overthinking it. There will be end products specifically for this and they’ll be trained by their peers / the company.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#180

Post by mike_hock »

metalman wrote: post #25

there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow for the…

2 misplaced apostrophes, 8 spelling errors — definitely human output
Post reply on HN