Detecting LLM-Generated Texts with “Classical” Machine Learning
blog.lyc8503.net
Detecting LLM-Generated Texts with “Classical” Machine Learning
1–10 of 184 posts
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#2today, sure.
Tomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.
Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#3I had done the same for classifying and generating bookmarks of thousands of datasheets, along with a very naive yolo-based classificator (to detect pages made out of diagrams and pictures mostly).
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#4The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ?
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
(the article was a good read, thanks!)
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#5The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ? If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads. (the article was a good read, thanks!)
I assume different models will have different distribution, so it has to be kept updated?
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#6I wonder about this technique vs simple SVM classifiers: https://x.com/rosmine/status/2056406399471558872?s=20
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#7The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ? If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads. (the article was a good read, thanks!)
I assume different models will have different distribution, so it has to be kept updated?
The article mentions that AI texts are often caught by multiple models, so hopefully text from newer LLMs could still be caught without updating the model?
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#8Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#9Anything too “clever” and “snappy” = instaLLM
Re: Detecting LLM-Generated Texts with “Classical” Machine Learning
#10Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading. Images, absolutely, there are…
It depends on how much text. For example, chardet often falls down on short strings, but 1K characters it nails it.