11–20 of 48 posts
Doesn't seem like a well-curated list. Some of these, e.g. Uber's God Mode, don't even fall under the normal usage of the term "AI". Others, like Palantir or WeChat, seem to denigrate an entire company (rather than specific practices). And the racist chatbot thing isn't even an actual product, nor was it intentionally designed to be racist.
Can't think of a racist AI scandal where is was intentional. The point of those stories is that it's easy to do accidentally.
Ehh.. the discrimination part sounds a lot like: "we had an AI look at statistics and it reported the patterns we choose to ignore".
AIs produce correlation which gets sold as causation.
A lot of these are less "woah, AI is real bad and we need to be better about using it appropriately" and more "people need to stop being fooled by snake oil products that promise impossible things."
You can blame users for expecting AI to solve everything, but that doesn't mean we give Engineers and Researchers free reign to package, publish, and distribute AI software that falls short of its promise ( and introduces dangerous patterns along the way).
Ehh.. the discrimination part sounds a lot like: "we had an AI look at statistics and it reported the patterns we choose to ignore".
COMPAS points out an uncomfortable trend ("violent crime is more prevalent in black communities...") but applies it in a way that is bad ("...and therefore black people should get longer criminal sentences for the same crime").
We both agree, I'm sure, that socioeconomics and race shouldn't meaningfully impact sentence length. We may also agree that it often does. What a lot of these models do is encode that outcome (that bias, some might say) in seemingly 'objective' systems. That reproduction of bias is why they're discriminatory - not because they look at crime stats, but because they apply them in a way that most people wouldn't.
A lot of these are less "woah, AI is real bad and we need to be better about using it appropriately" and more "people need to stop being fooled by snake oil products that promise impossible things."
You can blame users for expecting AI to solve everything, but that doesn't mean we give Engineers and Researchers free reign to package, publish, and distribute AI software that falls short of its promise ( and introduces dangerous patterns along the way).
And who is going to monitor and enforce this? You?
- 1. AI that's just not good enough, e.g. it requires 100% confidence not to be problematic (Detecting whether it's a person or a gorilla. Autonomous driving)
- 2. AI that's correctly trained but discovers problematic trends and attempts to do prediction based on these trends (Train on poverty + race correlations)
- 3. AI that's incorrectly trained on inherently biased data and just reaffirms the problematic trend (Train only on male CVs for hiring prediction.)
- 4. Buggy code that spits out bad data.