Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-driving cars less likely to detect children and people of color

businessinsider.com

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Self-driving cars less likely to detect children and people of color

#41

Post by strken »

Lewton wrote: post #6

they're testing 8 different detection algorithms > The detection systems were 19.67% more likely to detect adults than children, and 7.52% more likely to detect people with lighter skin tones than people with darker skin tones, according to the study. while they all had a harder time with adults vs children, that 7.52% is gotten by averaging 2 algorithms that performed abysmally, with 6 that had no statistically sign…

The two with significantly worse performance were RetinaNet and YOLOX. I don't really know anything about the field, but it's interesting they're both single stage performant models, while the slower but lower miss-rate RCNN variants are two-stage. It's interesting that the pedestrian-specific models are all worse than the general models at detecting people!

The conclusion is kind of weird: apparently their "findings reveal significant bias in the current pedestrian detectors" despite the bias being almost entirely within the single-pass general object detectors. And where it's statistically significant in the other models, the miss rate is low in both cases, and the effect is reversed! (Dry-weather Cascade-RCNN does better on dark-skin than light-skin, among others.)

Re: Self-driving cars less likely to detect children and people of color

#43

Post by elcano »

candiddevmike wrote: post #26
hermannj314 wrote: post #23

"A new technology reduces mortality risk for all people, but has slightly better outcomes for white adults." Conclusion - we call on lawmakers to make this technology illegal. We prefer more people die at equal rates more than we prefer less people to die at unequal rates. I am not sure I agree with the ethics that underlies this way of seeing the world.

It should be illegal, the current systems will never deliver without additional object detection methods (radar, lidar, etc). Get this shit recalled and back to the drawing board.

I wouldn't say 'never'. But the right thing to do is to keep the training wheels (radar, lidar) until it's proven that the system has learned to drive without them when there's good visibility and to reduce speed (or stop, if necessary) when the visibility is reduced. This also includes learning to recognize almost every possible situation, including zero shot events.

Re: Self-driving cars less likely to detect children and people of color

#44

Post by hermannj314 »

asddubs wrote: post #30
hermannj314 wrote: post #23

"A new technology reduces mortality risk for all people, but has slightly better outcomes for white adults." Conclusion - we call on lawmakers to make this technology illegal. We prefer more people die at equal rates more than we prefer less people to die at unequal rates. I am not sure I agree with the ethics that underlies this way of seeing the world.

What if the technology was only/primarily tested with white people, rather than inherently having better outcomes for white adults? It's not really as clear cut as you make it out to be, technologies at this level of complexity aren't just derived from physical/biological principles. Perhaps there was a better variant of the technology that was scrapped as a cost-cutting measure, because it performed the same for whi…

Fair point. Cost benefit analysis for adoption of safety features in automobiles is inconsistent.

Drunk drivers kill more Americans each few months than terrorists on planes have in the last 25 years. Yet every airline passenger must prove they aren't a terrorist but no one driving a car has a default presumption they are drunk. Unless you've been convicted previously, then maybe sometimes.

I wouldn't be surprised if a better model exists for object detection and we aren't using it to save pennies. Politics and ethics in automobile safety is asinine. Fair point.

Re: Self-driving cars less likely to detect children and people of color

#45

Post by yorwba »

Lewton wrote: post #6

they're testing 8 different detection algorithms > The detection systems were 19.67% more likely to detect adults than children, and 7.52% more likely to detect people with lighter skin tones than people with darker skin tones, according to the study. while they all had a harder time with adults vs children, that 7.52% is gotten by averaging 2 algorithms that performed abysmally, with 6 that had no statistically sign…

I think you misunderstood table 6. All algorithms show significant differences in miss rate for children, two show significant differences based on gender, and four others based on skin color. The four that showed no statistically significant difference between light and dark skin had very high miss rates overall. Of the other four, two are much worse for dark skin, and two are slightly better. Those last two are also best at detecting children, but 28% miss rate is still a bit too high for my taste.

Re: Self-driving cars less likely to detect children and people of color

#46

Post by jhot »

luma wrote: post #28
candiddevmike wrote: post #26

Earlier quoted context omitted.

It should be illegal, the current systems will never deliver without additional object detection methods (radar, lidar, etc). Get this shit recalled and back to the drawing board.

Do you have lidar and radar when you drive? Should it be illegal for you to drive without those things?

Why guess depth from images when you can know depth?

Re: Self-driving cars less likely to detect children and people of color

#47

Post by micromacrofoot »

sokoloff wrote: post #36
candiddevmike wrote: post #24

Earlier quoted context omitted.

The point of the article, to me, is computer vision will never be enough. These are machines and need to be augmented with radar and other object detection methods.

It’s intended to replace human drivers who have not yet evolved radar. I agree that radar could make it easier and/or more reliable, but there’s a pretty strong argument that building a system to equal/exceed humans using vision alone is possible.

Humans haven’t evolved radar, but computer vision systems are also worse than human vision systems in many ways, so they need to compensate

Re: Self-driving cars less likely to detect children and people of color

#49

Post by mrweasel »

zirgs wrote: post #8

How do they work in winter then? You can't see much skin if someone is wearing a winter coat. Right - self driving cars are a solution for Silicon Valley only only so they don't even bother testing those cars elsewhere.

The skin color is going to be the least of your issues in the winter time. How are the cars going to "see" the road under 10cm of snow? Granted humans shouldn't really be driving in those conditions either, but we do and mostly successfully, to avoid sleeping on the side of the road in -10C.

What is it that makes it so hard for all these algorithms to work on people with darker skin? This has been an issue for more than ten years, surely someone has started adding various skin colors into the training data. Is it a case of lack of training material, or is it just faster to focus on one skin type?

Re: Self-driving cars less likely to detect children and people of color

#50

Post by ben_w »

candiddevmike wrote: post #24
jl6 wrote: post #19

> While the study did not use the exact software companies like Tesla use to power self-driving cars because they are confidential, the software systems used for the study are based on the same open-source AI those companies use, according to Zhang. I was under the impression that commercial self-driving software was deeply proprietary and confidential, and there is no way to know that this study will generalize if r…

The point of the article, to me, is computer vision will never be enough. These are machines and need to be augmented with radar and other object detection methods.

Radar etc. almost certainly make it easier, but for it to be "never" this would also have to be provably impossible for humans.

Which isn't too say it's not never, as I remember studies in my own childhood that said human drivers were also bad at recognising how far away children were, and I've never heard of human perception of skin colour being tested in this way so it might just turn out that melanin is unfortunately good camouflage against tarmac…

…but unless and until that suggestion turns out to be correct of all humans, I default to assuming we're an existence proof of the capability to do without, and that means I still wouldn't say "never" to sufficiently advanced AI doing at least as well.

Post reply on HN