Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

aclu.org

Post reply on HN
Print view View original

101–110 of 114 posts

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#101

Post by andrewguenther »

ubernostrum wrote: post #82
andrewguenther wrote: post #76

> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist…

Claiming the algorithm is racist because your data is racist is inflammatory bullshit. Unless the designers of the system are aware of and account for biases in the training data, the system will be biased and will be biased due to the actions of its designers. I don't see what's so hard to accept about that. It's literally the oldest problem in computing (going back to the "Pray Mr. Babbage, if we put into your mach…

Is the claim that it is up to the system to detect bias in training data it is given?

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#103

Post by hoschicz »

spamizbad wrote: post #77
andrewguenther wrote: post #76

> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist…

How is stating a fact- it disproportionately matched people of color- somehow the same as claiming racism?

The fact is irrelevant to the problem and there is no reason to mention it, if they want the debate to stay professional and not emotional.

The whole racism angle of the article seems way too loaded for me.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#104

Post by stevew20 »

zo1 wrote: post #73
sudhirj wrote: post #62

Earlier quoted context omitted.

What does the link have to do with sketches? It's about name similarity on the no fly list. Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an…

" [..]conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south. " That is where the two sides of this argument diverge. First: That things "will start with guns drawn". And Second: That things are more likely to "go south" for an innocent individual incorrectly blamed by this.

Police have a poor history in the US of exercising trigger discipline. A fair amount of this is due to overly aggressive training that is designed to keep the officers as safe as possible, sacrificing safety of anyone perceived as a threat (real or not), but another large portion is caused by the culture of dominance and entitlement that is endemic to our law enforcement.

If police engage a target with a weapon already drawn, there is a long list of things that person can do that will get them shot, and a very short (and often unclear) list of actions that will keep them safe.

Source: 10 years PMO in the Marine Corps, 1 month SFPD (then quit because their training was so shitty)

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#105

Post by ubernostrum »

andrewguenther wrote: post #101
ubernostrum wrote: post #82

Earlier quoted context omitted.

Claiming the algorithm is racist because your data is racist is inflammatory bullshit. Unless the designers of the system are aware of and account for biases in the training data, the system will be biased and will be biased due to the actions of its designers. I don't see what's so hard to accept about that. It's literally the oldest problem in computing (going back to the "Pray Mr. Babbage, if we put into your mach…

Is the claim that it is up to the system to detect bias in training data it is given?

My personal claim is it's up to the people creating and training the system to be aware of biases and to account for that in how they create and train the system.

The problem, of course, is how woefully underprepared the average tech person is for realizing the existence of even extremely obvious biases, let alone more subtle and pernicious ones, and a tech culture in which people are lauded for being knee-jerk contrarians on topics like "does our society have a history of systemic racism".

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#106

Post by dang »

hanselot wrote: post #28
bunderbunder wrote: post #23

Earlier quoted context omitted.

It's relevant, but not as an excuse to dismiss the problem. One of the perennial problems with machine learning is that it has a tendency to intensify pre-existing biases in the system. It's not just that the algorithm tends to reflect biases in the source data, it's that that reflection tends to encourage the people using the system, who generally have some role in creating that source data in the first place, to be…

biasception. This can be the title of a new film where ai-powered echo chambers produce progressively more polarized societies until civil war and anarchy destroy all of humankind. Oh wait, this might just be reality already.

We've banned this account for repeatedly breaking the site guidelines. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future.

https://news.ycombinator.com/newsguidelines.html

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#107

Post by Maybestring »

dfundako wrote: post #47
alphabettsy wrote: post #46

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those a…

I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can b…

Malice is beside the point here is the it? The concern isn't an evil contractor, so much as an incompetent one.

Car manufacturers are sued for incompetent use of their products.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#108

Post by dragontamer »

AdmiralAsshat wrote: post #20

I don't see why this is a bad thing. Falsely arrest 28 members of Congress due to poor face recognition, and the problem of facial recognition in law enforcement is resolved the next day.

Congress doesn't stick up for each other. As soon as those 28-Congressmen are gone, they'll start passing legislation that those 28-people didn't want.

Congress is a battleground. Not a club or secret society. You send representitives from your state to try and get a slice of the pie, and I send my reps from my area to try and get a slice for me.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#109

Post by StanislavPetrov »

petermcneeley wrote: post #59
sudhirj wrote: post #58

Earlier quoted context omitted.

Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get…

What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...

Because most people don't view sketches with the sort of religious reverence they do technology.

Re: Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

#110

Post by StanislavPetrov »

Kalium wrote: post #87
mrweasel wrote: post #19

Earlier quoted context omitted.

Depending on the technologi behind Amazons facial recognitions, there's very real chance that the software simply associated some black facial features with "being criminal". If that's the case we're back to craniometry in the 1910-1940. Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a lar…

They fed it a large number of mugshots. They didn't ask "is this person a criminal". They asked it "does this face match one of the faces you've seen before". You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work. Systems like Rekognition let a user specif…

They didn't tell it to use 80%, 80% is the default setting. Anyone who thinks that many law enforcement agencies aren't going to use the default setting ought to go look at their routers and other hardware and see how they are configured.
Post reply on HN