“ The researchers now suggest that social-media platforms should incorporate chess language in their algorithms to avoid future incidents like this.” Or maybe don’t automate banning YouTube channels via nebulous black boxes? Or can advertisers stop being afraid of anything and everything?
Setting this case aside because it's sloppy reporting based on what another classifier found, this is a hard problem. Youtube gets too many videos and comments for humans to manually review them. Flagging helps, and Google likely uses ML more complex than just text and classifier (who is commenting, is the post trending, patterns in comments, etc.) When things do reach manual review, remember that humans make mistake…
- Ultimately, any supervised technique will depend on some system that can provide ground truth, which in these applications must be from humans. You can't avoid some humans in the loop and still learn a fuzzy human concept like "is hate speech". But if your aren't gathering that ground truth info in a very deliberate way, you can end up with a range of gaps or artifacts.
- There's a broad swath of technical approaches which would consider multiple kinds of information, but ensemble them together such that a very strong signal from one can have a determinative impact. It would be easy to have a situation where if the video itself talks about white/black and attack/threat/defend many times, the score from some language model into a top level ensemble model is very high and the other components may not matter.
- But a lot of those other components may often be in agreement because of natural structure. A video says black/white a lot, several other videos on the channel do the same, and a bunch of interested users also look at other channels that have very similar feature vectors, suddenly a content level, channel level and community level signal are all in agreement, and your model can believe it found a subcommunity of hate speech.
Incorporating many kinds of info into an ML model isn't that hard ... But doing so in a way which is tightly integrated (eg I'm likely to use a specific meaning of a term in my video which matches a use in a video of a related channel linked in my video description, and that use of that word is also related to what's visible in the video at around that time) is pretty hard. Doing so in a way which distinguishes predictions from confidences is harder.
All of which is just to say that it's hard to build a really good black box for a complex problem, especially one based around subjective human concepts, and maybe both we and the black boxes would do better by attaching more uncertainty to their predictions.
If it's worth the channel owner spending many hours making videos, and viewers spending many many more watching them, I think it's not crazy to ask that if the model predicts that it's likely disallowed content, that a manual reviewer can take a quick look.