I have been responsible a bit of content moderation myself, and interacted with mods on various forums. It's possible that this is responsible for some of my views, summarized here.
- The "secret" aspect of all this can be largely ignored. Twitter is under no obligation to tell its users exactly how their Tweets are moderated. I am sure most moderation tools are secret, including whatever Twitter (or for that matter, HN) uses today.
- There are aspects of user privacy that are important (e.g. how user data is gathered, where is it stored, whether it complies with various data governance regulations) which would be interesting, but have been largely ignored by these Tweets so far.
- I don't think Weiss understands what "shadow ban" means. It means that you post Tweets, but no one ever sees them. None of the practices she describes amount to what is usually described as a shadow ban or a hellban. Preventing a post from showing up in trends or searches is not a shadow ban. Google SafeSearch does not "shadow ban" explicit images or links.
- All companies have hierarchies of escalation. It is not surprising to me that there is a "top people" level of escalation in the form of SIP-PES that Weiss mentions. In fact, I'm reasonably sure something similar probably exists at Twitter today, consisting of Musk and some close advisors. They might even be using the exact same group!
- A lot of the discussion Weiss quotes as if it's top-secret information seems to show...individuals behaving reasonably while struggling with the extremely hard problem of content moderation at scale? Roth and others seem to be essentially saying, "We don't want to kick these users off the platform. Maybe if we limit their reach a bit, we won't get them inciting howling mobs that target and disturb other users?" Post-Musk Twitter seems to be struggling with the same thing, as the Kanye West debacle shows.
Overall, I don't really see too much that would cause me to gasp with horror. All I can really see is a company that never grew to Facebook scale trying to grapple with its content moderation problem without having the resources or perhaps temperament to hire hundreds of cheap content moderators like Facebook does.