Live data from Hacker News

SlopStop: Community-driven AI slop detection in Kagi Search

blog.kagi.com

111–120 of 271 posts

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#111
You'll probably have to think carefully about anti-abuse protection.

A great deal of LLM-generated content shows up in comments on social media. That's going to be hard to classify with a system like this and it will get harder as time goes on.

Another interesting trend is false accusations of LLM use as a form of attack.

Unlike other user-report detection (e.g. medical misinformation), this swims in the same direction as most AI misinformation. User-reported detection is typically going against the stream of misinformation by countering coordinated campaigns and pointing the user to a verifiable base truth. In this case there's no easy way to verify the truth. And the big state actors who are known to use LLMs in misinformation campaigns are battling the US for AI supremacy and so have an incentive to attack the US on AI since it's currently in the lead.

Especially if you're relying on volunteers, this seems prone to abuse in the same way, e.g. Reddit mods are. Thankless volunteer jobs that allow changing the conversation are going to invite misinformation farms or LLM farms to become enthusiastic contributors.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#113

Earlier quoted context omitted.

You’re responding to the Kagi ML lead. They are using those signals in addition to crowd sourcing.

Are you certain? I haven't seen this mentioned anywhere, except for now. And lot's of SEO WordPress spam is still showing up in Kagi queries.

Yes, I'm the ML lead.

The current search engine doesn't go after WordPress plugins we consider correlated to bad pages.

By far the most efficient method in the search engine for spam is downranking by trackers/javascript weight/etc.

Slopstop is going after page formats but we didn't plan to scale that back to rankings for everyone quite yet, only use it as features to detect AI slop. Otherwise the collateral damage on good actors with bad websites would be risky early on.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#114

You'll probably have to think carefully about anti-abuse protection. A great deal of LLM-generated content shows up in comments on social media. That's going to be hard to classify with a system like this and it will get harder as time goes on. Another interesting trend is false accusations of LLM use as a form of attack. Unlike other user-report detection (e.g. medical misinformation), this swims in the same directi…

> A great deal of LLM-generated content shows up in comments on social media.

True, but going after classifying the source (user's commenting patterns) is a better signal than the content itself.

That said, for us (Kagi) it's a touchy area to, say, label reddit comments as slop/bots. There's no doubt we could do it better than reddit (their whole comment history is only 6TB compressed) but I doubt *reddit* would be pleased at that.

And it's a growing issue for product recommendation searches -- see [1] at last section for example on how astroturfed reddit comments on product questions trickle up to search engine results.

> Another interesting trend is false accusations of LLM use as a form of attack.

Fair again, but the question of AI slop is much more about "who is using the tool how" than the content of the output itself.

Also we're looking to stay conservative. False negatives > false positives in this space.

> And the big state actors who are known to use LLMs in misinformation campaigns are battling the US for AI supremacy and so have an incentive to attack the US on AI since it's currently in the lead.

Not wrong, we're especially going after the deluge of low effort slop, and cleaning up the internet for our users.

Highly sophisticated attacks are likely to evade detection.

> Especially if you're relying on volunteers, this seems prone to abuse in the same way, e.g. Reddit mods are.

The human labelling/review aspect is expected to stay small and from trusted users.

The reporting is wide scale, but review is and will remain closed trust based group.

[1] https://housefresh.com/beware-of-the-google-ai-salesman/

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#115
post #59

Earlier quoted context omitted.

If your wife can't detect that you told your secretary to buy something nice, should she care?

This is an absurd comparison - you (presumably) made a commitment to your wife. There is no such commitment on a public blog?

Illuminating that you think the illustrated problem has something to do with a commitment.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#116

Earlier quoted context omitted.

Are you certain? I haven't seen this mentioned anywhere, except for now. And lot's of SEO WordPress spam is still showing up in Kagi queries.

Yes, I'm the ML lead. The current search engine doesn't go after WordPress plugins we consider correlated to bad pages. By far the most efficient method in the search engine for spam is downranking by trackers/javascript weight/etc. Slopstop is going after page formats but we didn't plan to scale that back to rankings for everyone quite yet, only use it as features to detect AI slop. Otherwise the collateral damage o…

> Yes, I'm the ML lead.

I never had any doubt about that ;)

What I was meaning with "are you certain" is regarding how Kagi treats the spam signals from WordPress plugins and themes. And now you gave the answer, thanks for that! I believe you will have good returns in using those signals.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#117

We wrote the paper on how to deslop your language model: https://arxiv.org/abs/2510.15061

Slop is about thoughtless use of a model to generate output. Output from your paper's model would still qualify as slop in our book. Even if your model scored extremely high perplexity on an LLM evaluation we'd likely still tag it as slop because most of our text slop detection is using sidechannel signals to parse out how it was used rather than just using an LLM's statistical properties on the text.

Would love to see proof of this claim that you can tag antislopped LLM text as LLM generated. I'm willing to bet money that you can't.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#118

HN could use some of this. It'd be nice if there was a safe having from the equivalent of high grade junk mail.

we just need human attestation. A vial of blood per comment

I can live with that ;)

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#119

Earlier quoted context omitted.

Slop is about thoughtless use of a model to generate output. Output from your paper's model would still qualify as slop in our book. Even if your model scored extremely high perplexity on an LLM evaluation we'd likely still tag it as slop because most of our text slop detection is using sidechannel signals to parse out how it was used rather than just using an LLM's statistical properties on the text.

Would love to see proof of this claim that you can tag antislopped LLM text as LLM generated. I'm willing to bet money that you can't.

I'm not saying we could detect it from the text alone!

The side channel signals (who posted it, where, etc.) are more valuable in tagging than raw text classifier scores.

That's why I said our definition of slop can include all types of genAI: it's about *thoughtless use of a tool* more than the tool being used.

And also that regardless of the method, your model can be used to generate slop.

Re: SlopStop: Community-driven AI slop detection in Kagi Search

#120

I always wondered if social networks ran spamd or spamassassin scans on content…though I’m not sure how effective a marker that tech is today. This obviously is more advanced than that. I just turned this on, so we shall see what happens. I love searching for a basic cooking recipe so maybe this will be effective.

Give it time, the database is just starting. Give it ~2 weeks to start seeing real impact on your results

Thanks. Your company is proof the search problem has plenty left to solve for. Looking forward to this.
Post reply on HN