Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting LLM-Generated Texts with “Classical” Machine Learning

blog.lyc8503.net

Post reply on HN
Print view View original

71–80 of 184 posts

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#71

Post by WhitneyLand »

driverdan wrote: post #18
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

There are two problems, false positives and changing the LLM's pattern. It's really easy to have a false positive and false positives can be very harmful if the person using the detector isn't aware of that risk. It's also very easy to change the pattern of LLM output. You can provide basic prompting that will significantly change the structure of the output. For example, having it utilize the Wikipedia article on si…

"It's really easy to have a false positive"

Not really. The false positives for the SOTA detector are very very low.

"It's also very easy to change the pattern of LLM output."

Not in a way that can reliably avoid detection. The problem is the patterns are baked into the distribution itself. It's smoothed over, so it becomes difficult to prompt your way out of that.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#72

Post by hgoel »

connorboyle wrote: post #65

> Eventually, I faked my way through the thesis, and life moved on. This is a very startling admission! I checked the Chinese (original?) version of the post, and saw the author uses the word "糊弄" (in the place of "faked"); I'm not a native speaker but I think this may come across more as a self-effacing comment on the low quality and/or effort behind their thesis, whereas the English version implies fraud. May be wi…

I don't know if the Chinese text implies something different, but I think even in English it's pretty normal for people to claim they 'faked' their way through something without referring to fraud.

E.g. "I faked my way through the interview!" = "I did my best to respond to questions I did not feel fully prepared for, and managed to get through the interview"

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#73

Post by pixl97 »

stymaar wrote: post #66
pixl97 wrote: post #52

Earlier quoted context omitted.

I mean, back when I was spam filtering setting up a simple Bayesian classifier was easy. Train it on your spam and ham and it worked damned good. "Mission Accomplished".... until it wasn't. Spam rates started climbing and it started getting harder than ever to filter them. There is always an incentive to get spam to bypass filters, so as your filters increase in accuracy, those attempting to pass said filters adjust…

I don't think it's a very good remark, as there's significantly less email spam than 20 years ago. Another example is ad-blocker-blocker. There was a little bit of an arm race between ad blockers and advertisers in the middle of the 2010s, but it didn't last long. Advertisers mostly just decided not to care about ad-blockers.

>Advertisers mostly just decided not to care about ad-blockers.

Directly not to care because they lost in court.

And yet the biggest advertizer on Earth (Google) decided to change their browser to make adblocking far more difficult. That or they say "just use an app, oh and turn on notifications". I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.

There is significantly more spam than 20 years ago, just less of it reaches your inbox. This is a very important distinction as the cost of spam filtering is just as high as ever. On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them. This allows these companies to have an overwhelming influence on email on the internet, to the point they can send spam with near impunity, and where if your system does it will be nuked from orbit by their systems.

And much like now Google supplies both the email spam, and the solution to the spam, they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#74

Post by onecomment1 »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

This sounds like it was edited by an llm.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#75

Post by mr_mitm »

Wowfunhappy wrote: post #63
docheinestages wrote: post #24

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing detail…

…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated. If I can do it, an algorithm should be able to do it. Maybe in the future the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.

How do you estimate your false negative rate?

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#76

Post by pishpash »

Wowfunhappy wrote: post #63
docheinestages wrote: post #24

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing detail…

…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated. If I can do it, an algorithm should be able to do it. Maybe in the future the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.

What's likely to happen though, is human idiomatic writing will degrade to AI level and the two will converge. Just like nowadays it is harder to tell if some (human) non-native English speaker wrote some English text online whereas 20 years ago it was very obvious.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#77

Post by overgard »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

I don't know, the thing about most text slop is how little effort goes into disguising it (for now, anyway). I'm sure anyone dedicated can go undetected, but it's the really low-effort stuff that's generally the problem. If you can catch some of it, that's something at least.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#78

Post by jaco6 »

akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

The best method is, as always, an anti-privacy method.

Simply track all citizens' writing patterns throughout their life, from cradle to grave, then diff with any given text's signature--you'll know if it was human written or not.

Better--opt in--install a "personal text signature" on your devices, sign things that you wrote yourself with it.

But I suppose that's just like the image provenance chips on cameras.

Either way father fascism is more with us than ever, praise him!

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#79

Post by jaco6 »

40four wrote: post #26

I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solv…

[dead]

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#80

Post by hgoel »

WhitneyLand wrote: post #67
akersten wrote: post #8

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…

"Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it...it's a bad fiction to perpetuate that any of this is anything more than tarot card reading." Not true at all. Pangram is highly effective and has a very low false positive rate. The post here is impressive for a small project, it looks like they independently thought of one of the core ideas Pangram uses of…

So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work?

If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?

Post reply on HN