Self-driving cars less likely to detect children and people of color

Post by tbrownaw » Mon, Aug 28, 2023, 4:34 PM UTC

wongarsu wrote: post #66

ben_w wrote: post #50

Earlier quoted context omitted.

Radar etc. almost certainly make it easier, but for it to be "never" this would also have to be provably impossible for humans. Which isn't too say it's not never, as I remember studies in my own childhood that said human drivers were also bad at recognising how far away children were, and I've never heard of human perception of skin colour being tested in this way so it might just turn out that melanin is unfortunat…

I'm confident that a comprehensive study would show that on average humans are worse at detecting people of color against dark backgrounds (if we agree this is code for "people with highly pigmented skin" and not Asians or Latinos). There is just much less contrast to work with, and dark skin also makes facial features stand out less (which is an issue because faces are the thing humans can recognize best). There is…

> But of course the ideal case is much easier to define, and has fewer things that make some people uncomfortable.

As another bonus, it also provides an extra excuse to advocate against replacing humans with AIs.

Post by foooorsyth » Mon, Aug 28, 2023, 5:24 PM UTC

mola wrote: post #88

foooorsyth wrote: post #31

Earlier quoted context omitted.

> Feels like a case of outrage-for-clicks Like 99% of these “AI discrimination” articles. >human-detecting AI is developed in a western country with ~60% white population. Most of the training data is collected there >the AI performed slightly worse in Uttar Pradesh, where the people and everything else in the background look different >AI is prejudiced! Get outraged! Every time.

Weird, you articulated the exact point of bias in AI but the tone you used is dismissing. Yes obviously AI is not a moral agent and it isn't racist per se. But if it's input is biased and the test is biased then the application will be biased. That's a problem, if you go and deploy these models where their training data is lacking. Let's say, self driving car using the AI you described deployed in Uttar Pradesh is le…

I'm dismissive of endless streams of unhelpful clickbait articles written by barely-tech-literate journalists aiming to spark racial outrage. I'm not dismissive of the threat of bias in AI, and I'm certainly not a fan of cavalier automotive companies running clearly-alpha autonomous software on public roads.

Still, I find most of these kinds of articles to be obnoxious and unhelpful in their shaming. Did you read the paper linked in the OP? It investigates the following datasets: CityPersons, EuroCityPersons, and BDD100k.

* CityPersons data is from mostly Germany (with all of it coming from central Europe)

* EuroCityPersons, as the name implies, is data from European cities

* BDD100k data is from NYC and Bay Area

So are we shocked with the outcomes here, or are they more or less obvious? If I trained a popular object detector with pedestrian image data solely collected in India, would you be surprised if it performed poorly outside of India? Would that warrant a racially-inflammatory article title?

And, as others have pointed out, these types of investigations make the implicit assumption that autonomous companies aren't working hard to reduce these biases in their own internal training pipelines. As reckless as some of those companies are, I promise you that they are not solely relying on Western data before deploying to non-Western streets. The investigators here may have simply been lazy/biased themselves (only investigating openly-available datasets from Western regions), and then projecting that laziness/bias onto AV companies.

Re: Self-driving cars less likely to detect children and people of color

Re: Self-driving cars less likely to detect children and people of color