Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

aclu.org

Post reply on HN
Print view View original

81–90 of 114 posts

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#81

Post by spamizbad »

dfundako wrote: post #35
fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

LE doesn't have the best track record following best practices. For example, they still rely heavily on notoriously unreliable eye-witness accounts and polygraph tests, two things science has largely debunked.

They were slow to let go of phrenology as well.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#82

Post by ubernostrum »

andrewguenther wrote: post #76

> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist…

Claiming the algorithm is racist because your data is racist is inflammatory bullshit.

Unless the designers of the system are aware of and account for biases in the training data, the system will be biased and will be biased due to the actions of its designers.

I don't see what's so hard to accept about that. It's literally the oldest problem in computing (going back to the "Pray Mr. Babbage, if we put into your machine wrong figures, will the right answers come out" days).

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#83

Post by stretchwithme »

People have similar faces. Who knew?

The technology is not perfect. And should never be used as evidence of a crime, just as an indication that two images might be of the same person and to have humans look at them.

And then those humans will also make mistakes. People do look alike. And there are twins.

I think a jury should require more evidence than just similar appearance. But such a match is a strong indication of where to look.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#84

Post by ocfx »

delecti wrote: post #49
dfundako wrote: post #35

Earlier quoted context omitted.

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

I would not be shocked at all if some LE somewhere used the default settings. It seems absurd to assume otherwise. It's not as though LE has a great track record. https://en.wikipedia.org/wiki/No_Fly_List#False_positives

LE won't be given "settings"

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#85

Post by BigChiefSmokem »

andrewguenther wrote: post #76

> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist…

And the data is "racist" as well because...?

This whole topic is racially charged because incarceration rates are a race issue and you are being disingenuous with your comment pretending the problem with facial recognition can somehow exist and be fixed in isolation.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#86

Post by dastbe »

yread wrote: post #50
cthalupa wrote: post #15

https://www.theverge.com/2018/7/26/17615634/amazon-rekogniti... >Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Presented witho…

On the other hand if you want to judge whether it would work on 300 million people with 95% confidence then running on 535 members of congress with 80% confidence might be a good approximation

(AWS employee)

That’s not how these numbers fit together. Using rekognition, you would create a collection of faces based off your image corpus and then ask the service if a provided image has a match against anything in your collection. Lowering your confidence threshold, invariant of your sample size, is by definition going to increase your false positive rate while lowering your false negative rate.

I think it would be worthwhile to compare a mixed test set and see false positives/negatives at various confidence thresholds, because the fear (IMO) is that law enforcement will minimize false negatives at the expense of a huge increase in false positives.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#87

Post by Kalium »

mrweasel wrote: post #19
joemaller1 wrote: post #10

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

Depending on the technologi behind Amazons facial recognitions, there's very real chance that the software simply associated some black facial features with "being criminal". If that's the case we're back to craniometry in the 1910-1940. Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a lar…

They fed it a large number of mugshots. They didn't ask "is this person a criminal". They asked it "does this face match one of the faces you've seen before".

You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work.

Systems like Rekognition let a user specify what level of certainty the system has that it has found a match. In this instance, the system was told to use 80%. Rekognition can be told to use any number from 0 to 100. Your number of 99.999% could be used without any difficulty whatsoever! Since the ACLU doesn't seem to be sharing enough to reproduce the test, we can't tell what would happen.

Regulation based on accuracy is a wonderful idea! One well worth pursuing. With that said, "accuracy" is perhaps difficult to define here. How would you determine that? How accuracy is measured is highly sensitive to what you use as a dataset, and different test datasets can easily produce very different impressions of how common false positives and false negatives are.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#88

Post by pandasun »

aarong11 wrote: post #11

Could this be down to the people training the algorithm being predominantly middle class white people? It's a pretty well known phenomenon that people are bad at recognising features of people of other races.

>middle class

? That doesn't add up to the rest of your comment.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#89

Post by see-dante »

stretchwithme wrote: post #83

People have similar faces. Who knew? The technology is not perfect. And should never be used as evidence of a crime, just as an indication that two images might be of the same person and to have humans look at them. And then those humans will also make mistakes. People do look alike. And there are twins. I think a jury should require more evidence than just similar appearance. But such a match is a strong indication…

I don't think that's the point -- the problem isn't that "oh there are other processes that [...]," it's that this wave of technology is misused, is bias, and care must be taken when any of the signals we create.

This is especially important as automation enters the equation.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#90

Post by akira2501 »

sadris wrote: post #38
mrweasel wrote: post #31

Earlier quoted context omitted.

The problem is that it's only 95% perfect in this case (28 out of 534, being identified as criminals). That a pretty big margin of error when the result is possible arrest. The scope of its use is also in question, sure if you looking a someone who kidnapped a child, 95% is good enough. If you use it for any minor violation you risk harassing a large percentage of the population, who then will need to prove that it w…

They weren't "identified as criminals." They were identified as possibly being criminals . (And as other posters have shown, they were identified with only 80% confidence as being a criminal.) And an officer can review the evidence presented by the report and make a human determination to follow up with a physical arrest.

> They were identified as possibly being criminals.

And you've highlighted the _exact_ problem here. That is not at all what has happened, they have been identified as "possibly having a similar appearance to one or more particular images of a suspected criminal."

> And an officer can review the evidence presented by the report and make a human determination to follow up with a physical arrest.

Which already presumes that the system will only have false positives. When the system produces false negatives, then this mode of investigation is entirely flawed.

Post reply on HN