Why do companies with unbounded resources still have terrible moderation?

"I said, I’m telling you, you’re not getting the billion dollars. I said, you’re not getting the billion. I’m going to be leaving here in, I think it was about six hours. I looked at them and said: I’m leaving in six hours. If the prosecutor is not fired, you’re not getting the money. Well, son of a bitch. (Laughter.) He got fired. And they put in place someone who was solid at the time."

Apparently I’m a nazi, because I believe in liberty. Are you advocating this for me?

Case in point - youtube. They changed their demonitization algo in response to complaints and now they have complaints from other users that the algo is producing false positives. For example, the "Great War" history podcast had 100s of youtube videos demonitized because they talk about storm troopers and Nazis, which of course isn't great if you are a Nazi, but maybe ok if you're a history podcast.

The issue is that each video generates on an individual level so little money for youtube that it just doesn't pay to have an individual go through each and every video on their platform and make a value judgement on worth. I mean anyone can upload a video for free and each watch of a video is maybe at maximum a few cents from an advertiser in profit to youtube.

Facebook and all other content aggregation web forums face the same conundrum. And the problem doesn't get better if we invent new algos or people more efficiently (somehow). On the one hand you now have an algo that determines what can be said or not said en masse and is making the decisions in an increasingly black box (as the heuristics will be by necessity rather complicated), on the other you have the very poor filtering videos or at the very least fallible human judgement still at the wheel on what gets to be said and not said.

Who watches the watchers? At the moment we have a handful of powerful media conglomerates that effectively control through monopsony almost all the social media on the internet. What, we've got Alibaba, Facebook, youtube, reddit, and instagram - that covers at a rough guess like 60-70% of social media traffic? But these are companies that are only successful at scale - you go on a social media platform because all your friends are on it (a la the myspace model) and you are given a "free" user experience because aggregating millions of people allows for a few cents in ad revenue per user to subsidize the servers at scale.

This problem is probably baked into the cake of this particular pattern and isn't fixable. Boy is it going to be wild when we finally crack how to make deep fake videos!

In Albert Hirschman’s typology it’s all exit, voice and loyalty are irrelevant.