Earlier quoted context omitted.
The point is because sometimes even a perfectly reasonable inference from an ML model would be considered a big mistake due to societal considerations that are unknown to the model. For example, a couple years ago, there was a big hubbub over a Google Image labeler that labeled a black man and woman as "gorillas". A mistake for sure, but the headlines about the algorithm being "racist" were wrong. The algorithm was c…
The actual racist thing is that humans who don’t consider or prepare for or include affected people in deciding to deploy models trained to produce racist outcomes. It doesn’t matter that the machine has no opinions, it matters that the machine produces outcomes reflecting harmful biases. Banning the word doesn’t change that, but neither does treating the biased process as unbiased.
It can be wrong or right but it is not making a judgment based on anything outside of math.
You are correct to say the training wasn't complete but that doesn't mean anyone did anything wrong, racist, or hateful... 99% of the time it's simply a mistake.
When you label things like that as racist instead of simply mistakes you water that word down to the point where it becomes meaningless.
The problem in the last 10+ yrs of outrage internet social justice is that in order to gain attention and get traction those involved have lumped so many things into terms like racism that that it eventually becomes so stretched it's meaningless.