Earlier quoted context omitted.
The problem is that one is explainable and the other is not. "He was near the scene of the crime and has no convincing alibi," is very different from, "The computer said he's a suspect and we don't know why, but we still want a warrant." People being targeted for being anomalous is bad, but centralizing and scaling it up is worse.
> The computer said he's a suspect and we don't know why, but we still want a warrant. Mission critical ML systems (G/FB ads, crime forensics, medical decision support, financial algorithms) do not work like this. The designers know exactly which features are causing responses/predictions in the model, their respective perturbation sensitivities, and have clear bounds regarding adversarial inputs/outliers. “Not knowi…
Are you speaking from experience or from intuition?