Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men • Hacker News Board

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#131

Post by hodgesrm » Sun, Sep 05, 2021, 12:19 AM UTC

jpambrun wrote: post #124

hodgesrm wrote: post #109

Earlier quoted context omitted.

Glad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding. The interesting question is how the hell do we fix this?

One example doesn't support your claim that QA was not sufficient. You will always get misclassifications, most won't be offensive to anyone. This is not a medical device or a self driving car, this is Facebook trying to squeeze more engagement..

This type of misclassification is offensive and for good reason. Similar errors have happened regularly for a long time and in many different systems. [1]

The comments about enlarging training sets illustrate the blinders that many technical people have about this technology. There does not seem to be a notion of independent quality assurance to detect what should by now be obvious problems.

[1] https://www.bostonmagazine.com/news/2018/02/23/artificial-in...

Edit: oops, left out source

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#132

Post by Redoubts » Sun, Sep 05, 2021, 1:31 AM UTC

jrm4 wrote: post #100

Black person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tech" and by extension here. And even as I write this, I'm not trying to judge -- just more open the door to understanding, like, y'all understand this fact, right? I find myself searching for metaphorical equivalents. I'm thinking something like "what if it label…

Now imagine the care taken when applying this technology to dating sites, and other social services; for clustering or recommendation engines, or prediction services. This whole field is a joke, that refuses any introspection.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#133

Post by jrm4 » Sun, Sep 05, 2021, 2:25 AM UTC

jpambrun wrote: post #124

hodgesrm wrote: post #109

Earlier quoted context omitted.

Glad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding. The interesting question is how the hell do we fix this?

One example doesn't support your claim that QA was not sufficient. You will always get misclassifications, most won't be offensive to anyone. This is not a medical device or a self driving car, this is Facebook trying to squeeze more engagement..

I have never seen a more sadly oblivious statement than this: the suggestion that Facebook squeezing engagement is significantly less dangerous than malfunctioning medical devices or self driving cars.

Facebook is more dangerous; precisely because the harm in getting it wrong is so much easier to hide.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#134

Post by jrm4 » Sun, Sep 05, 2021, 2:31 AM UTC

nzeribe wrote: post #128

suifbwish wrote: post #121

Earlier quoted context omitted.

I also happen to be black but that actually has very little bearing on what I have to say and am merely mentioning it to prevent anyone from suggesting that my opinion on the matter is related to my skin color. The problem with this is no one actually trained the AI to have a racially bias but rather did not train it on recognizing the difference. Why? Probably because no one knew the computer was going to identify t…

I find it baffling and dispiriting that you, as a Black person, believe this is more likely to be an "error" than an act of deliberate racism. It's getting silly, now. A white racist programmer has done this on purpose because he finds it hilarious. The chances of a computer equating monkey and man, given the correct datasets, are close to nil. There's a blindness - a wilful ignorance - to look away and show good fai…

Yup. Though I would refine it a bit -- it's the "rush to excuse" that's the most dangerous. I don't much worry about the deliberate racist programmer as much as I worry about subconscious deliberate-or-not biases creeping in.

This perfectly analogizes to most everywhere else; angry open n-word saying racists are usually powerless losers.

More problems come from the larger combination of the rest of the "racism" spectrum, whether apathetic, or racist-but-quiet, or harbors latent biases that they may not know about etc.

(And here I do feel like I have to say, the answer isn't "SMOKE THEM OUT AND EXPOSE THEM" on the personal level, it's just taking the utmost care in the work you do)

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#135

Post by jpambrun » Sun, Sep 05, 2021, 2:52 AM UTC

nzeribe wrote: post #128

suifbwish wrote: post #121

Earlier quoted context omitted.

I also happen to be black but that actually has very little bearing on what I have to say and am merely mentioning it to prevent anyone from suggesting that my opinion on the matter is related to my skin color. The problem with this is no one actually trained the AI to have a racially bias but rather did not train it on recognizing the difference. Why? Probably because no one knew the computer was going to identify t…

I find it baffling and dispiriting that you, as a Black person, believe this is more likely to be an "error" than an act of deliberate racism. It's getting silly, now. A white racist programmer has done this on purpose because he finds it hilarious. The chances of a computer equating monkey and man, given the correct datasets, are close to nil. There's a blindness - a wilful ignorance - to look away and show good fai…

As someone who worked on ml based classification I can totally see a man being misclassified as a monkey on occasion. If you have man and monkey in the output vector, it will happen on from time to time.. If you find a way to achieve 100% accuracy, go claim your Nobel prize..

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#136

Post by ant6n » Sun, Sep 05, 2021, 6:11 AM UTC

jrm4 wrote: post #100

Black person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tech" and by extension here. And even as I write this, I'm not trying to judge -- just more open the door to understanding, like, y'all understand this fact, right? I find myself searching for metaphorical equivalents. I'm thinking something like "what if it label…

I find a lot of the discussion here baffling and tone deaf. There are a lot of comments saying "but humans are primates". It's a dismissal of systemic racism issues coming from a privileged position that's akin to responding the black lives matter movement with "all lives matter".

Some of the commenters may exactly think that the algorithm is objective, forgetting that it's built by ppl, for ppl, tested by ppl. If the algorithm kept labelling bearded white dudes as vaginas, it wouldn't be pushed to production, because it would found as a bug and dealt with somehow. But labelling people of color as monkeys is apparently something they're unaware of, which doesn't necessarily point to deliberate racism, but a systemic racism issue.

In a lot of discussion here, everyone, on all sides of the argument, is getting voted down. Perhaps this shows this community is not ready for these types of discussions.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#137

Post by skinkestek » Sun, Sep 05, 2021, 7:12 AM UTC

pc86 wrote: post #126

skinkestek wrote: post #122

Earlier quoted context omitted.

White person here: At this point, if a web scale service like this do this it should be grounds for significant sanctions: increasing fines (think GDPR scale) and closer scrutiny (think mandatory inspections and quality controls every month, paid for by the company) should do I think. But, while I'm really annoyed by this and support you I'm also sure there will be enough outrage against it in time from.

Fined for what, exactly? They deployed a flawed AI model. The flaw was pointed out, and they removed the model and apologized. I'm genuinely curious what else should have happened.

What should have happened is they did basic testing up front?

And since they fixed it right away small fines only, like nuisance small. And small follow up.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#138

Post by Melchizedek » Sun, Sep 05, 2021, 8:14 AM UTC

jjj123 wrote: post #112

rscoots wrote: post #108

Earlier quoted context omitted.

Interested in hearing how you obtained the proprietary training material for this algo from Apple. Can we see it? Also which teams at apple made the algorithm? Since you claim to know they were all 'white'. In reality, I'm sure everyone working on this team was well aware of these potentials for training set bias before most of us even knew what ML was. Even if this was considered it'd be pretty risque to have an emp…

I worked closely with an AI ethicist at $100B+ tech company company and their findings were that all of our AI training data was biased and the ML teams had no plans to de-bias their training data because they had no incentive to do so. I have no reason to believe this finding doesn’t hold for the FAANG companies. AI ethics are mostly an afterthought in tech today. And to your second point about how would we know if…

Tech is not disproportionately white. It’s disproportionately Asian.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#139

Post by nzeribe » Sun, Sep 05, 2021, 12:16 PM UTC

jpambrun wrote: post #135

nzeribe wrote: post #128

Earlier quoted context omitted.

I find it baffling and dispiriting that you, as a Black person, believe this is more likely to be an "error" than an act of deliberate racism. It's getting silly, now. A white racist programmer has done this on purpose because he finds it hilarious. The chances of a computer equating monkey and man, given the correct datasets, are close to nil. There's a blindness - a wilful ignorance - to look away and show good fai…

As someone who worked on ml based classification I can totally see a man being misclassified as a monkey on occasion. If you have man and monkey in the output vector, it will happen on from time to time.. If you find a way to achieve 100% accuracy, go claim your Nobel prize..

But that's not what happened, is it? The algorithm hasn't miscategorised men, it has miscategorised Black men. I'd argue the AI has done its job, with a high degree of accuracy. It found what the programmers were looking for when they wrote the code. It was meant to be an insult, and was highly effective. Lack of outrage on HN is as expected; the authors are quite possibly in the comments holding their sides with laughter.

Re: Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men

#140

Post by jpambrun » Sun, Sep 05, 2021, 12:47 PM UTC

nzeribe wrote: post #139

jpambrun wrote: post #135

Earlier quoted context omitted.

As someone who worked on ml based classification I can totally see a man being misclassified as a monkey on occasion. If you have man and monkey in the output vector, it will happen on from time to time.. If you find a way to achieve 100% accuracy, go claim your Nobel prize..

But that's not what happened, is it? The algorithm hasn't miscategorised men, it has miscategorised Black men. I'd argue the AI has done its job, with a high degree of accuracy. It found what the programmers were looking for when they wrote the code. It was meant to be an insult, and was highly effective. Lack of outrage on HN is as expected; the authors are quite possibly in the comments holding their sides with lau…

Those are baseless accusations based on poor understanding of ML.

This behaviour is counterproductive. It is not susceptible to sway me towards your point of view, but instead discredit you and those holding similar opinions.