Autospam and Naive Bayes
pixelfed.blog
Autospam and Naive Bayes
1–10 of 39 posts
Re: Autospam and Naive Bayes
#2TL;DR, without the dramatic narration:
Naive Bayes is an old spam filter that works on words independently from each other, ie not assuming any grammatical or semantical sense to what it processes.
It makes it simple, but efficient as a first line of defense against "dumb keyword stuffing spam".
(No LLM used, if that ever means anything these days).
Re: Autospam and Naive Bayes
#3Naive Bayes is also surprisingly powerful for deduplication of large datasets of messy data (and the related problem of record linkage).
It's still competitive with cutting edge approaches if carefully applied, and usually much faster so it can be applied to big data.
It's the engine behind Splink, the free Python library that I develop: https://github.com/moj-analytical-services/splink
Re: Autospam and Naive Bayes
#4The Bayes algorithm was core of one of my first shared software projects - a local spam filter plugin for the blog engine serendipity. The surprising thing back then was how well it worked in practice, the surprise now is that it still works just fine.
It's used to filter blog comments since over a decade now.
And I mean that with the surprise. It was tought to us as the simple method used before, and I was just testing whether it maybe can still be of use a bit. Mainly because it was easy to understand, it just counts, and I liked the position with the a priori probabilities. «Definitely still useful» was not the expected outcome.
Re: Autospam and Naive Bayes
#5TL;DR, without the dramatic narration: Naive Bayes is an old spam filter that works on words independently from each other, ie not assuming any grammatical or semantical sense to what it processes. It makes it simple, but efficient as a first line of defense against "dumb keyword stuffing spam". (No LLM used, if that ever means anything these days).
> Naive Bayes is an old spam filter that works on words independently from each other
It should be pointed out that a lot of ML models operate on a 'bag of words' approach where words (tokens) are considered without regard for structure/order. Bag of words is surprisingly powerful. It can be expanded to include n-grams (n words in order) and this makes it a powerful technique.
Punctuation (where punctuation is counted as a token) is a crazy powerful predictor in text classification including spam.
Re: Autospam and Naive Bayes
#6TL;DR, without the dramatic narration: Naive Bayes is an old spam filter that works on words independently from each other, ie not assuming any grammatical or semantical sense to what it processes. It makes it simple, but efficient as a first line of defense against "dumb keyword stuffing spam". (No LLM used, if that ever means anything these days).
True. It can be extended to work also on bigrams instead of individual words, I'm not sure how often this is done in spam filters.
For an in-depth explanation of how it works, I suggest the excellent book from Jurafsky and Martin https://web.stanford.edu/~jurafsky/slp3/4.pdf
Re: Autospam and Naive Bayes
#7TL;DR, without the dramatic narration: Naive Bayes is an old spam filter that works on words independently from each other, ie not assuming any grammatical or semantical sense to what it processes. It makes it simple, but efficient as a first line of defense against "dumb keyword stuffing spam". (No LLM used, if that ever means anything these days).
Yeah a good next easy step could be to use word2vec and build a quick linear regression model.
Re: Autospam and Naive Bayes
#8I predicted this more than 2 years ago when I was running a fedi node.
Back in 2004 I used to build spam filtering proxies using bayesian filters in spamassassin. And my experience in the fedi instantly made me realize they would be a quick and easy solution for a lot of fedi spam.
So it's fun to see pixelfed taking the leap. I can't wait to see how it evolves.
The fedi being very similar to e-mail is most likely going to see a lot of the developments of e-mail. For example domain verification would also be good. And I also predict some sort of reputation system for instance domains.
Re: Autospam and Naive Bayes
#9See also the Paul Graham classics "A Plan For Spam" (2002) [0] and "Better Bayesian Filtering" (2003) [1].
Always love to see "simple" 20yo ideas find a new use case.
Re: Autospam and Naive Bayes
#10pixelfed is doing remarkable work in fediverse, this just appears to be one of those things that other fediverse projects will adopt pretty quickly