Detecting LLM-Generated Texts with “Classical” Machine Learning • Hacker News Board

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#181

Post by Wowfunhappy » Fri, Jul 17, 2026, 7:24 PM UTC

mr_mitm wrote: post #158

Wowfunhappy wrote: post #87

Earlier quoted context omitted.

No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair. Edit: But I'm also super conflicted about this, because I really want to read what humans think, not what an AI thinks, regardless of the writing quality.

My point is that you cannot know that AI generated text is obviously AI generated. It's like the old "hair pieces look awful, I can tell immediately!" - no, you can only tell when they're awful. Maybe you're not as good as you think at detecting AI.

Well, but I think my false positive rate is very low, it’s just the false negative rate that could potentially be quite high. This bothers me much less than the reverse.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#182

Post by joouha » Mon, Jul 20, 2026, 9:13 AM UTC

lcnhk42 wrote: post #176

cygn wrote: post #105

Earlier quoted context omitted.

I built a browser extension that does this, well for posts on twitter, hackernews, reddit etc. If you want it for all text, it would also be feasible. I use a quantized mini-LM model that runs very fast and classifies eg your whole twitter feed in a couple of seconds. Check it out: https://slopsieve.com/extension Accuracy is also much higher than this approach here. 0.9944 AUC, 0.966 acc@.5, 0.971 F1@.5

Hey this is really cool. Do you mind sharing a little bit about your approach? Is it similar SVM like algorithm or something else entirely?

Not GP, but I too was interested in this slopdetect-minilm-v3 model.

The model appears to be similar to MiniLM-L6 (384 dimension, 6 transformer layers) but uses RoBERTa/GPT-2 style embedding/tokenization (50265 vocab size), used for binary classification, and quantized to INT8.

My guess is that is was distilled from roberta-base, then fine-tuned on freely available pile like artem9k/ai-text-detection-pile or similar.

It's a nice model - I've just used it to create a browser extension which highlights text based on how likely it is to be LLM genereated.

Edit - a quick google search reveals ibm-granite/granite-embedding-30m-english with the same architecture: slap a binary classification head on, fine tune, job done.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#183

Post by joouha » Mon, Jul 20, 2026, 3:25 PM UTC

joouha wrote: post #182

lcnhk42 wrote: post #176

Earlier quoted context omitted.

Hey this is really cool. Do you mind sharing a little bit about your approach? Is it similar SVM like algorithm or something else entirely?

Not GP, but I too was interested in this slopdetect-minilm-v3 model. The model appears to be similar to MiniLM-L6 (384 dimension, 6 transformer layers) but uses RoBERTa/GPT-2 style embedding/tokenization (50265 vocab size), used for binary classification, and quantized to INT8. My guess is that is was distilled from roberta-base, then fine-tuned on freely available pile like artem9k/ai-text-detection-pile or similar.…

Just noticed in the model's metadata:

> MiniLMv2 L6-H384 (30M params) progressively distilled L24→L12→L6 from RoBERTa-large-v8 teacher (neobert-v2 recipe, min_words=0). Byte-level BPE tokenizer. INT8 quantized.

Re: Detecting LLM-Generated Texts with “Classical” Machine Learning

#184

Post by Der_Einzige » Fri, Jul 24, 2026, 5:02 AM UTC

WhitneyLand wrote: post #163

Der_Einzige wrote: post #123

Earlier quoted context omitted.

As min-p approaches 1, the temperature you can get away with approaches infinity. Also more modern samplers like top-n-sigma are explicitly designed to get away with temperature of infinity.

Not convinced just using top n sigma is going to beat a SOTA detector. The fingerprints Pangram uses should in principle be able to detect style above the token selection level. Other papers have tried to beat it with temperature and it didn’t work, although I haven’t seen anyone try insane levels. Give it a shot and let me know if you have any success.

It's easier for you to test my assertions than it is for me to test yours, unless you're willing to give me ~5000+ USD in my Pangram account.