Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

nytimes.com

Post reply on HN
Print view View original

101–110 of 173 posts

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#101

Post by Y_Y »

6gvONxR4sf7o wrote: post #58

There’s an amazing number of “but people are primates” comments. Correctness here includes specificity. If person and primate are both candidates, then primate isn’t a correct answer to what this is trying to do. You wouldn’t react to the phrase ‘treating women as objects’ by saying people are objects, right?

You make a good point, and I'd agree that it's implicit that they mean "non-human primates". Otherwise they could solve this by labelling lots of humans as "primates" in the dataset.

On the other hand, I think there's a semantic difference between your two cases. I'd argue that treating a woman as an "object" is a different sense of the word to how women (and men and teapots) are literally "objects".

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#102

Post by dagmx »

Y_Y wrote: post #97
Tagbert wrote: post #68

“Aren't people primates?” Words have meaning on multiple levels. On a technical level, yes, all humans are primates, but if the algorithm were following this rule, it would tag all humans in all photos as primates, but it didn’t do that. It only tagged black people. That takes us to a social meaning level. In this context, you have to consider the long, racists history of dehumanizing black people. They have often be…

I've dealt with a lot of badly labelled datasets, but I can't say I ever saw one maliciously labelled by racists. There are other more subtle problems that may contribute. As an amateur photographer I've been told that most cameras are calibrated for light skin. If you aren't careful with your setup you'll get bad definition in shadows and relatively very dark areas. If you have badly calibrated/exposed photos then v…

This is usually a good example of societal bias compounding.

It's possible that nobody directly involved in this was actively racist, but we're building on centuries of racism in every facet. So a little bit of bias in each technology and data set involved, not to mention the engineers and their testing, just adds up

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#103

Post by ourmandave »

ratsmack wrote: post #52
ourmandave wrote: post #21

Earlier quoted context omitted.

Did they have even one black person on the AI team? https://www.credera.com/insights/racial-bias-in-machine-lear...

Why would that have made a difference?

Geez, I dunno.

If only it was explained in a conveniently linked article.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#104

Post by tartoran »

diskzero wrote: post #89

I would like to know if the model was poorly trained or if this was caused by a malicious individual modifying the algorithm to generate offensive suggestions.

I think the technology is very limited but instead it’s overused in cases that shouldn’t. No matter what model is used and how much it’s trained, we have to come to our senses and realize that it will never be perfect and should’t be used everywhere. It’s shudders me to think that this technology is used in legal cases to make life affecting decision that could be absolutely wrong

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#105

Post by tartoran »

zaptrem wrote: post #63
jl6 wrote: post #57

This particular AI failure mode keeps surfacing, but seems to be fixed rapidly when it generates bad press. How do they actually correct it? Teach it some more granular categories of living things? Slap a sensitivity filter on top of its output?

AFAIK Google just blocked all “gorilla, primate” related tags in Google Photos.

Aha, a patch.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#106

Post by petre »

fallingknife wrote: post #71
ant6n wrote: post #60

Earlier quoted context omitted.

The problem is the training set, which apparently skews too much towards white ppl, and doesn't include checks for the primates/non-white humans differentiation. One could argue this is due to white people building things for white ppl, which does indeed point towards systemic racism in the overall development process.

But the problem is that there isn't any primates/non-white humans differentiation because humans are primates. I doubt anybody smart enough to be an AI engineer of any race would care if they were called a primate, because they are.

Does the subject have clothes on? Most probably not a monkey.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#107

Post by tartoran »

amanaplanacanal wrote: post #53
tgv wrote: post #27

Earlier quoted context omitted.

It's unlikely. Search for images of primate, and you'll see fairly dark, humanoid shapes. Poor training data then seems like a more probable reason for visual confusion. > You only have a story worth reading when it’s against an underprivileged minority. To some extent. But I think there would also be a story if some medicine would work badly on white people, or men. The problem here is that these labels would be the…

Yeah, white primates are pretty weird. It only happened because a few migrated north into temperate Europe and Asia and lost their melanin a while ago.

It would be fine for Black people too if the context was purely taxonomy and all people would be classified as primates. But that is not what happened here.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#108

Post by rscoots »

ant6n wrote: post #60
hn_throwaway_99 wrote: post #55

Earlier quoted context omitted.

OK, I'll try to tread carefully here, but I think it's really helpful to separate the technical issues from the societal issues. In the Google photo case, I've seen the picture in question, and it's not difficult an all to see how a rudimentary statistical algorithm would mistake the photo for a gorilla. No, no human would make this mistake, but due to the lighting in the photo and the woman's hair it's really not ha…

The problem is the training set, which apparently skews too much towards white ppl, and doesn't include checks for the primates/non-white humans differentiation. One could argue this is due to white people building things for white ppl, which does indeed point towards systemic racism in the overall development process.

Interested in hearing how you obtained the proprietary training material for this algo from Apple. Can we see it?

Also which teams at apple made the algorithm? Since you claim to know they were all 'white'.

In reality, I'm sure everyone working on this team was well aware of these potentials for training set bias before most of us even knew what ML was.

Even if this was considered it'd be pretty risque to have an employee dedicate time specifically to differentiating black ppl from apes in the algorithm.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#109

Post by hodgesrm »

jrm4 wrote: post #100

Black person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tech" and by extension here. And even as I write this, I'm not trying to judge -- just more open the door to understanding, like, y'all understand this fact, right? I find myself searching for metaphorical equivalents. I'm thinking something like "what if it label…

Glad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding.

The interesting question is how the hell do we fix this?

Post reply on HN