Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

Post by vorpalhex » Thu, Jul 26, 2018, 2:44 PM UTC

dfundako wrote: post #47

alphabettsy wrote: post #46

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those a…

I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can b…

Post by mc32 » Thu, Jul 26, 2018, 2:46 PM UTC

Post by exabrial » Thu, Jul 26, 2018, 2:54 PM UTC

Post by philipodonnell » Thu, Jul 26, 2018, 3:14 PM UTC

dfundako wrote: post #35

fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

Post by petermcneeley » Thu, Jul 26, 2018, 3:37 PM UTC

sudhirj wrote: post #58

joemaller1 wrote: post #10

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get…