[deleted]
You either die an MVP or live long enough to build content moderation
61–70 of 429 posts
Re: You either die an MVP or live long enough to build content moderation
#62Earlier quoted context omitted.
Yeah, then someone has to create or find that whole table and make. The initial problem wasn't those symbols but the content itself, the symbols and special characters came into the problem later. Later on as mentioned in my original comment, that they would use positive content from other blog posts that were published/passed the moderation to mix up their bad content. Probably could use a different method, but at t…
The good news is that the Unicode consortium has a report on this issue, and the tables already exist for normalization and mapping of confusables to their ASCII lookalikes: https://www.unicode.org/reports/tr39/
Oh, that's nice.
I guess I can use that next time time to work on the data cleaning for that model.
Thanks.
Re: You either die an MVP or live long enough to build content moderation
#63I'm building a human content moderation service as an API for developers. It's not fully ready yet (expected next week) but people can sign up and start exploring the docs. I'd love to hear your feedback and any features you might want to see:
Re: You either die an MVP or live long enough to build content moderation
#64Earlier quoted context omitted.
If your eyes can "normalize" a unusual symbols to a common one to make an English word then so can a lookup table. I feel like this isn't a case where you'd reach first for a neural net.
In fact, the Unicode consortium provides a report and extensive list of "confusable" symbols, which you could use alongside Unicode normalization tables to map adversarial back into more ASCII-equivalent text before running it through anti-spam mechanisms that are interested in the content of the message. https://www.unicode.org/reports/tr39/
I built a Python library for finding strings obfuscated this way. Was critical when moderating our telegram channel before an ICO.
https://github.com/wanderingstan/Confusables
E.g. "𝓗℮𝐥1೦" would match "Hello"
Re: You either die an MVP or live long enough to build content moderation
#65I'll never forget having to be a moderator for a somewhat popular forum back in the day and oh man did I learn how a few people can make your life hell. One thing not mentioned many times in these discussions are the poor moderators. Having to look at all that stuff, some of which can be very disturbing or shocking (think death, gore, etc as well as the racy things) really takes a toll on the mind. The more automatio…
> I'll never forget having to be a moderator for a somewhat popular forum back in the day Similar experience, though I'll say that the worst was dealing with other teenagers that threatened suicide when you banned them. That always took a lot of effort to de-escalate and was a complete drain on personal mental health. I could deal with porn, shock images, and script kiddie defacements, but having people threaten to k…
Several years ago a popular gaming forum with a significant teenage audience I used to read had declared a simple policy toward threats of suicide. If you were threatening to kill yourself, do it, and stop messaging the mods, they are not here to talk you down from a ledge. It seemed pretty effective.
Re: You either die an MVP or live long enough to build content moderation
#66The internet itself is unmoderated in any useful sense for content, yet it has lived longer than most of these cheesy "moderated" products that seek to impose their morality on you.
The "internet" isn't liable, so moderate is in the form of transparent traffic shaping. When disruptions are small, costs are either absorbed in aggregate by infrastructure owners (and user attention) until traffic is literally moderated away with routing.
Re: You either die an MVP or live long enough to build content moderation
#67Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…
How did the lousy content affect your legit users?
Every blog is pretty isolated from other users.
It's not that the content will be popped up to everyone when someone posted something.
There's a Feed page where you can read what others you follow have published.
There's a Explore page where latest content without any filter or categorization would be visible. This is where such content would appear, but only blogs older than 7 days would appear there (we have removed that delay in recent versions).
Basically no one noticed them.
Although we disclosed the issue we were dealing with some of the old users of platform when they complained about their posts not getting published. That was the first issue in first 15 minutes of the machine learning model classifying wrongly due to being fed mixed content (where bad content was mixed with good content from those exact blogs by spammers.)
Other than several bloggers reporting their posts wouldn't go through as expected, no one else got effected and I hope no people were lured with those scammer while their content was published on our platform.
Re: You either die an MVP or live long enough to build content moderation
#68So true.
In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like.
So here we are with this painful problem.
I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without the land of bots and dishonest users and so forth. But that obviously brings its own issues.
Re: You either die an MVP or live long enough to build content moderation
#69Earlier quoted context omitted.
In fact, the Unicode consortium provides a report and extensive list of "confusable" symbols, which you could use alongside Unicode normalization tables to map adversarial back into more ASCII-equivalent text before running it through anti-spam mechanisms that are interested in the content of the message. https://www.unicode.org/reports/tr39/
I built a Python library for finding strings obfuscated this way. Was critical when moderating our telegram channel before an ICO. https://github.com/wanderingstan/Confusables E.g. "𝓗℮𝐥1೦" would match "Hello"
I should have had better eyes while searching, could find this and saved some hours.
Re: You either die an MVP or live long enough to build content moderation
#70If you host blobs for free, somebody is going to use you as their host. Even if you just hosted audio, I'm sure somebody will quickly come along with a steganography tool to hide their content on your site (and use your bandwidth). Similarly, if you make compute power available, people will use you to mine cryptocurrency. Even if all you host is text, somebody will come along to be abusive. When you put a computer on…
> If you host blobs for free, somebody is going to use you as their host. Even if you just hosted audio, I'm sure somebody will quickly come along with a steganography tool to hide their content on your site (and use your bandwidth).
This feels like something more of a theoretical example cited versus something that has happened. Do you have any examples of steganography being used as bandwidth redirection/hosting?