Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

aclu.org

Post reply on HN
Print view View original

61–70 of 114 posts

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#61

Post by HeyLaughingBoy »

petermcneeley wrote: post #59
sudhirj wrote: post #58

Earlier quoted context omitted.

Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get…

What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...

One major difference is that non-technical people tend to trust what a computer says far more than they should. People are familiar with sketches and can make judgments on how clearly they match a face. They have no similar frame of reference for computer face recognition.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#62

Post by sudhirj »

petermcneeley wrote: post #59
sudhirj wrote: post #58

Earlier quoted context omitted.

Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get…

What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...

What does the link have to do with sketches? It's about name similarity on the no fly list.

Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an 89% match with a known killer, the conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south.

You really don't want to be scanned by a trigger happy cop having a bad day. God knows who you happen to look like from that angle.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#64

Post by Kalium »

jerf wrote: post #16
joemaller1 wrote: post #10

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion. At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad ide…

I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#65

Post by dragonwriter »

notyourwork wrote: post #60
fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

Threshold is a configuration property no different than a dynamo read capacity. Misconfigured properties don’t work the way you expect. I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

> I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

It can say exactly what the Amazon response to the Verge dismissing the ACLU test should, and, in fact, it should say that if Amazon really does have such critical, specific domain-specific recommended settings, rather than it being something made up off the cuff to mitigate bad press.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#66

Post by [deleted] »

alphabettsy wrote: post #48
sadris wrote: post #18

Internet used to be delivered through unreliable 28.8kbps modems. I didn't think for a second that we should cease using the Internet. Bone marrow transplants for cancer treatment used to have a 20% failure rate. I didn't think for a second that we should cease using bone marrow transplants. And yet why does the ACLU think we should cease using a technology to deliver potential location hits on wanted criminals becau…

I’m pretty certain they take issue with law enforcement using the technology that’s not 100% and without oversight because they have the ability to seriously alter someone’s life.

[deleted]

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#67

Post by Kalium »

notyourwork wrote: post #60
fluxsauce wrote: post #29

Earlier quoted context omitted.

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

Threshold is a configuration property no different than a dynamo read capacity. Misconfigured properties don’t work the way you expect. I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

If you're someone with minimal understanding of the use and deployment of technologies, who knows only of this technology as something used to detect faces, then it's a glaring oversight.

If you're a researcher making a deliberate choice to use the defaults to simulate a random police officer with no real understanding of their tools, then say as much. And compare with more cautious results.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#68

Post by guitarbill »

dfundako wrote: post #47
alphabettsy wrote: post #46

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those a…

I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can b…

It's still ethically questionable. In fact I'm struggling to come up with a better example than facial recognition tech (except other mass surveillance). Maybe cutting corners while developing driverless cars that results in the death of a pedestrian.

Almost every engineering discipline has a code of ethics [0][1][2][3]. It's time software "engineering" grew up and did the same.

I rarely see ethics mentioned on HN, and granted, people's view differ. But it's weird we're not having that conversation at all.

[0] https://www.raeng.org.uk/policy/engineering-ethics/ethics

[1] https://www.ieee.org/about/corporate/governance/p7-8.html

[2] https://www.nspe.org/resources/ethics/code-ethics

[3] http://www.asce.org/code-of-ethics/

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#70

Post by Kalium »

guitarbill wrote: post #68
dfundako wrote: post #47

Earlier quoted context omitted.

I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can b…

It's still ethically questionable. In fact I'm struggling to come up with a better example than facial recognition tech (except other mass surveillance). Maybe cutting corners while developing driverless cars that results in the death of a pedestrian. Almost every engineering discipline has a code of ethics [0][1][2][3]. It's time software "engineering" grew up and did the same. I rarely see ethics mentioned on HN, a…

You're right! This is a critically important conversation that we absolutely need to have within our profession. It's very often ignored and there's no support system for people who take ethical stands.

So. Let's talk about ethics. I, personally, subscribe to the ACM code of ethics.

I think Rekognition, as built and presented, falls fully within that strict ethical code. It can be put to uses that are unethical, but that does not fall upon the people who made it. Certainly, an engineer creating a system such as the one the ACLU created would be acting unethically.

Post reply on HN