You either die an MVP or live long enough to build content moderation

Post by mr_toad » Wed, Sep 29, 2021, 12:49 AM UTC

wanderingstan wrote: post #64

jffry wrote: post #54

Earlier quoted context omitted.

In fact, the Unicode consortium provides a report and extensive list of "confusable" symbols, which you could use alongside Unicode normalization tables to map adversarial back into more ASCII-equivalent text before running it through anti-spam mechanisms that are interested in the content of the message. https://www.unicode.org/reports/tr39/

I built a Python library for finding strings obfuscated this way. Was critical when moderating our telegram channel before an ICO. https://github.com/wanderingstan/Confusables E.g. "𝓗℮𝐥1೦" would match "Hello"

Post by PerkinWarwick » Wed, Sep 29, 2021, 1:16 AM UTC

srfvtgb wrote: post #248

PerkinWarwick wrote: post #169

Earlier quoted context omitted.

What I'd like is something identical to Facebook (preferably hosted in a box on my desk) that I invite all my friends to, and they have the ability to invite people to. 2nd gen is probably far enough. Anyone cuts up rough, I go by their house and make them miserable.

You lose out on the network effect with this solution. If someone gets to pick who's allowed onto a network, then they won't bother using it. Maybe your friends will join since they are allowed to add _their_ friends, but those friends of friends wouldn't bother because most of their friends can't join, meaning your friends won't bother either unless they really want to talk to you specifically. It would work for gro…

Post by SilverRed » Wed, Sep 29, 2021, 1:26 AM UTC

jfengel wrote: post #55

If you host blobs for free, somebody is going to use you as their host. Even if you just hosted audio, I'm sure somebody will quickly come along with a steganography tool to hide their content on your site (and use your bandwidth). Similarly, if you make compute power available, people will use you to mine cryptocurrency. Even if all you host is text, somebody will come along to be abusive. When you put a computer on…

Post by lmm » Wed, Sep 29, 2021, 2:00 AM UTC

tdeck wrote: post #58

Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

For adversarial problems like this, a shadowban approach can sometimes be necessary. Perhaps people can still see their blogs but GoogleBot gets blocked from indexing them, or they only appear to someone with the spammer's cookies. That way it takes them longer to catch on and evade the model. Of course, that means you'll need to at least spot check your bans because you can't rely on legit users escalating to you.

Post by nemo44x » Wed, Sep 29, 2021, 2:06 AM UTC

snowwrestler wrote: post #131

seph-reed wrote: post #121

Earlier quoted context omitted.

I still theorize crowd-moderated platforms are possible, as long as there's really good gate-keeping. My bet is some real-world tie, one which is time consuming and expensive to create. From there it should be possible to create moderation tools that keep the rest going. An example of a real world tie would be a trust network that requires status with in-person communities and local businesses. And not just "accept t…

Slashdot’s meta-moderation system worked well for a long time. One set of people could make moderation decisions directly on content, and then another unrelated set of people would review the moderation decisions and support or revert them. It was all tied to karma and permissions in ways I can’t quite remember. But essentially there was no way for a motivated bad-faith group to both moderate and meta-moderate themse…