Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

aclu.org

Post reply on HN
Print view View original

41–50 of 114 posts

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#41

Post by adamson »

raverbashing wrote: post #22
cthalupa wrote: post #15

https://www.theverge.com/2018/7/26/17615634/amazon-rekogniti... >Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Presented witho…

Thinking of an Onion headline: "This tech sucks" says person trying to use niche technology for the first time without knowing its specificities and limitations.

I guess the question is: do we trust any operator to know the specificities and limitations? I for one wouldn't trust Joe Cop to use something like this with the degree of discretion or understanding of statistical nuance called for here.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#43

Post by Jweb_Guru »

dfundako wrote: post #35
fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

Assuming that law enforcement is going to spend any amount of time being careful about enforcing correct use of forensic tools is hard to reconcile with the history of the American judicial system. For example, there is essentially no scientific evidence that a polygraph test measures anything at all, but polygraph evidence is considered admissible in many courts and used to be admitted in many more. Moreover, rigorous studies of fingerprint evidence have found them to have false positive rates as high as 1 in 18, but they are still often treated as close to infallible in court. I am not sure why you think facial recognition is going to be different.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#44

Post by bunderbunder »

hanselot wrote: post #28
bunderbunder wrote: post #23

Earlier quoted context omitted.

It's relevant, but not as an excuse to dismiss the problem. One of the perennial problems with machine learning is that it has a tendency to intensify pre-existing biases in the system. It's not just that the algorithm tends to reflect biases in the source data, it's that that reflection tends to encourage the people using the system, who generally have some role in creating that source data in the first place, to be…

biasception. This can be the title of a new film where ai-powered echo chambers produce progressively more polarized societies until civil war and anarchy destroy all of humankind. Oh wait, this might just be reality already.

Society's honestly not becoming particularly polarized, on anything remotely like a historical scale. To take US history, for example, I can think of several times in the past century alone that the public sphere was more acrimonious.

I haven't made much of a study of primary sources, but I wouldn't be surprised if a common thing that happened every one of those times was that people would start trying to figure out how to turn "appeal to acrimony" into a convincing debate tactic.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#46

Post by alphabettsy »

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those are real considerations for any program that has the potential to help ruin someone’s life.

It also possibly highlights the fact that algorithms are not immune to bias when they are designed by humans. Obviously I don’t think this bias is intentional, but so much of it isn’t and happens anyway.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#47

Post by dfundako »

alphabettsy wrote: post #46

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those a…

I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can be used improperly and have disastrous results, like a car or gun), could Amazon or a tech administering it be liable for someone falsely imprisoned?

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#48

Post by alphabettsy »

sadris wrote: post #18

Internet used to be delivered through unreliable 28.8kbps modems. I didn't think for a second that we should cease using the Internet. Bone marrow transplants for cancer treatment used to have a 20% failure rate. I didn't think for a second that we should cease using bone marrow transplants. And yet why does the ACLU think we should cease using a technology to deliver potential location hits on wanted criminals becau…

I’m pretty certain they take issue with law enforcement using the technology that’s not 100% and without oversight because they have the ability to seriously alter someone’s life.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#49

Post by delecti »

dfundako wrote: post #35
fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

I would not be shocked at all if some LE somewhere used the default settings. It seems absurd to assume otherwise. It's not as though LE has a great track record.

https://en.wikipedia.org/wiki/No_Fly_List#False_positives

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#50

Post by yread »

cthalupa wrote: post #15

https://www.theverge.com/2018/7/26/17615634/amazon-rekogniti... >Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Presented witho…

On the other hand if you want to judge whether it would work on 300 million people with 95% confidence then running on 535 members of congress with 80% confidence might be a good approximation
Post reply on HN