You either die an MVP or live long enough to build content moderation

It drives me absolutely nuts when I encounter a video platform upstart that has not adequately prepared (or prepared at all) for the inevitable onslaught of undesirable and illegal content that users will soon start uploading if the platform has really any traction at all. No UGC site/app is immune. Even when prepared, it is an eternal, constantly-evolving battle as users find more clever ways to try to hide their uploads or themselves. If you aren't ready for it at all, you may never be able to catch up. And while a lot of the undesired content could just be really annoying to get rid of, some is catastrophic -- a user uploading a single video of something like child porn that is publicly visible can be the death knell for the platform.

I’m going to go ahead and refute some of the counterarguments I’ve heard a million times over the years just to get it out of the way.

“It could be a while before it’s necessary.”

People seeking to upload and share unsavory content are constantly getting kicked off every other platform for doing so, and thus are always on the lookout for something new to try where they might be able to get away with it, at least for now. They are the earliest adopters imaginable.

“Just let users flag content”

Lots of issues here, but here’s a couple big ones.

1. You cannot afford something like child porn to be visible long enough to be flagged, or for it to be seen by anyone at all. If something like this gets uploaded and is visible publicly, you could be screwed. I worked on a video platform once that had been around a couple years and was fairly mature. One video containing child porn managed to get uploaded and be publicly visible for about one minute before being removed. It was a year before the resulting back-and-forth with federal agencies subsided and the reputation of the platform had recovered.

2. People uploading things like pirated content tend to do so in bulk. You might see people uploading hundreds of videos of TV shows or whatever. It may exceed legitimate uploads in the early days of a platform. You do not want to burden users with this level of moderation, and actually they aren’t likely to stick around anyway if good videos are lost in a sea of crap that needed to be moderated.

“We’ll just use (some moderation API, tool, etc.)”

Yes, please do, but I’m not aware of anything that works 100%. Even if you filter out 99% of the bad stuff, if the 1% that gets through is kiddie porn, say goodnight. These tools get better all the time, but users who are serious about uploading this kind of stuff also continue to find new and interesting ways to trick them. As recently as 2017 a pretty big video platform I worked on was only able to stop everything with a combination of automated systems as well as a team overseas that literally checked every video manually. (We built a number of tools that enabled them to do this pretty quickly.)

Content shouldn’t be moderated

Child porn? Hundreds of pirated episodes of Friends instead of legitimate user videos? (Even if you are pro-piracy, you don't want to pay to host and serve this stuff, and you don't want it to distract from legit original content from your users.) What about when some community of white supremacists gets wind of your new platform and their users bomb it with all their videos?

Do not take this stuff lightly.

EDIT: I've spent most of the last decade as an engineer working on UGC and streaming video platforms

There is no correct way to do the content moderation, so on my website, https://roastidio.us I don't plan to have any. My rules are simple:

* you can say whatever you want to say, but only one person, the one you replied to, get to see it other than youself

* the one you replied to get to approve whether it can be shown to other people. No further reply is allowed until this one is approved.

* Even the one you replied to don't approve your comment, the comment will not be deleted and will still be visible to you.

Even without seeing that stuff, seeing a constant stream of bad behaviors with the probably-good behavior filtered out can subtly change your priors about people - it makes you start thinking people suck more in general, kind of like how watching news where they show the worst of the worst makes one trust people less.

I definitely used to notice this after some time working on our moderation queues.

Hmm I wonder if it picked up timestamps as its initial filter.

I only learned about it myself after spending too long building my own half-baked version. I think it's in pretty opaque language that makes it hard to find even if you know what you want.

Maybe somebody else on here will see it and learn about it before they need it, and at least you still have a new tool to reach for in the future.