Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Photos AI still can't label gorillas after racist errors

theregister.com

Post reply on HN
Print view View original

41–50 of 74 posts

Re: Google Photos AI still can't label gorillas after racist errors

#41

Post by Sparkyte »

version_five wrote: post #2

I agree that blocking the feature is the smartest thing to do if people are going to get offended. It's idiotic to call it "racist", it's a shitty classifier, and it's incredibly irresponsible (and probably racist) as well as dismissive of actual racism to pretend that's racist. At the same time, I'm surprised they can't get this classifier to work. It doesn't seem like a very challenging problem (I'd consider myself…

I would take being called a gorilla a compliment. They are jovial, endearing and typically well tempered. Humans on the otherhand I would almost consider an insult. Since we start massive wars, raise cultures and poke fun at each other in ways that are racist. Kinda bad when animals are better role models.

But really it is history that is to blame. Gorillas are great. But if you called anyone a Lion or Lioness it would be perceived as a compliment. Gorillas don't deserve such connotation.

Re: Google Photos AI still can't label gorillas after racist errors

#42

Post by creato »

rjh29 wrote: post #21
edgyquant wrote: post #7

Earlier quoted context omitted.

How is that racist? Because you’re projecting a racist comment onto it, a classifier? That does not make sense to me

The racism comes from the fact non-white people were not properly considered when the model was developed and trained. This comes up time and time again in AI, ranging from face ID that only works on white people, to porn classifiers that associate black people with NSFW images.

No matter how good or well trained on good data with good representation of all skin colors a classifier is, it’s going to misclassify people and things periodically, and it’s definitely going to misclassify black people as gorillas more often than other races.

Re: Google Photos AI still can't label gorillas after racist errors

#43

Post by lisasays »

xeromal wrote: post #32
lisasays wrote: post #26

Earlier quoted context omitted.

If a system consistently misclassifies persons black persons far more than white persons -- and does so in a way that's obviously provocative and offensive -- then by definition it's racist in its effect (regardless of intent). The fact that the smartest company in the world cannot seem to get a handle on this problem after 8 years is also not unreasonable grounds to suspect that something's up. Like that they don't…

These are dangerous grounds to discuss, but I don't think it's racist (colloquially) at all. If gorillas were like yetis and covered in white fur and it started labeling anglos as gorillas, it's not racist either. Racism (colloquially) comes from bad people's intentions. Who would've thought that a creature that is very similar to us humans and has a color that matches some humans would accidentally classify somethin…

Racism (colloquially) comes from bad people's intentions.

Racism can also be measured by its effect, regardless of intent.

What would be racist from this outcome is if it kept doing this and no one did anything.

After 8 years, that's seems to be precisely what's happening.

Re: Google Photos AI still can't label gorillas after racist errors

#44

Post by cookieperson »

jmull wrote: post #31
version_five wrote: post #2

I agree that blocking the feature is the smartest thing to do if people are going to get offended. It's idiotic to call it "racist", it's a shitty classifier, and it's incredibly irresponsible (and probably racist) as well as dismissive of actual racism to pretend that's racist. At the same time, I'm surprised they can't get this classifier to work. It doesn't seem like a very challenging problem (I'd consider myself…

> It's idiotic to call it "racist", it's a shitty classifier, and it's incredibly irresponsible (and probably racist) as well as dismissive of actual racism to pretend that's racist. It might be useful to understand why it's offensive... Likening black people to apes/gorillas/monkeys is a racist trope with a long history. It's part of casting black people as less than human, which has been used to justify their ensla…

This. Once a bias is known it must be corrected. Allowing it to stand is basically saying "we don't care about black consumers, they can't use this feature, and we will continue to insult them on accident". Which it's no longer an accident, it's a known racially discriminative software feature...

It's like hiring someone to do plumbing. A few customers invite them into their home and report that the new plumber said racist stuff. As a business owner you do what after this?

Re: Google Photos AI still can't label gorillas after racist errors

#45

Post by changoplatanero »

rjh29 wrote: post #21
edgyquant wrote: post #7

Earlier quoted context omitted.

How is that racist? Because you’re projecting a racist comment onto it, a classifier? That does not make sense to me

The racism comes from the fact non-white people were not properly considered when the model was developed and trained. This comes up time and time again in AI, ranging from face ID that only works on white people, to porn classifiers that associate black people with NSFW images.

But, white people get misclassified as animals by the classifier too. Typically white people aren't misclassified as gorillas but as other animals. So i don't think the cause is as simple as non-white people not being considered during training.

Re: Google Photos AI still can't label gorillas after racist errors

#46

Post by lisasays »

prepend wrote: post #27
jjevanoorschot wrote: post #5

Earlier quoted context omitted.

I'm on board with assuming that the intent of Google / the engineers who built the classifier is not racist. However the outcome — labelling a black person as a gorilla — certainly is racist. What makes you think otherwise?

The reason I don’t think the actual act of the program bugging is racist is I imagine it’s just some stupid rule based on shape (primates have a similar shape) and skin color and isn’t smart enough to distinguish humans vs gorillas vs Gumby toys with black/grey skin closer to a gorilla. I think intent is important for labeling something racist and a function doesn’t have intent. And it doesn’t seem like the programme…

The reason I don’t think the actual act of the program bugging is racist is I imagine it’s just some stupid rule based on shape (primates have a similar shape) and skin color and isn’t smart enough to distinguish humans vs gorillas vs Gumby toys with black/grey skin closer to a gorilla.

But this is AI -- so the stupid rule is not pre-programmed, but rather curve-fit to the data (uh, "learned").

So ultimately it's a matter of (1) failing to find the right training data (or procedure) and (2) more fundamentally, choosing not to correct the problem after 8 years.

Re: Google Photos AI still can't label gorillas after racist errors

#47

Post by xeromal »

lisasays wrote: post #43
xeromal wrote: post #32

Earlier quoted context omitted.

These are dangerous grounds to discuss, but I don't think it's racist (colloquially) at all. If gorillas were like yetis and covered in white fur and it started labeling anglos as gorillas, it's not racist either. Racism (colloquially) comes from bad people's intentions. Who would've thought that a creature that is very similar to us humans and has a color that matches some humans would accidentally classify somethin…

Racism (colloquially) comes from bad people's intentions. Racism can also be measured by its effect, regardless of intent. What would be racist from this outcome is if it kept doing this and no one did anything. After 8 years, that's seems to be precisely what's happening.

Isn't the point of the article that it just refuses to recognize gorillas outright? That prevents exactly what you're talking about. And I made that point in my post. It is hurtful so Google prevent google photos from classifying anything as a Gorilla is a good bandaid. Some things are just too risky to solve for little gain.

Re: Google Photos AI still can't label gorillas after racist errors

#48

Post by tornato7 »

spiderfarmer wrote: post #15

I read the entire wikipedia page on racism to see why people call a bug in an experimental product feature, based on bad classifiers and data, a form of racism. It's unfortunate. And I can see why it worries people. Because on the surface it looks like a systemic problem. But is it really?

Perhaps it is not racism that the bug exists, but failing to find, fix, or disable a feature that makes offensive classifications is racist.

Re: Google Photos AI still can't label gorillas after racist errors

#49

Post by rjh29 »

changoplatanero wrote: post #45
rjh29 wrote: post #21

Earlier quoted context omitted.

The racism comes from the fact non-white people were not properly considered when the model was developed and trained. This comes up time and time again in AI, ranging from face ID that only works on white people, to porn classifiers that associate black people with NSFW images.

But, white people get misclassified as animals by the classifier too. Typically white people aren't misclassified as gorillas but as other animals. So i don't think the cause is as simple as non-white people not being considered during training.

It classified 80 photos of the same black person as 'gorilla', I have not heard of that happening with white people.

Re: Google Photos AI still can't label gorillas after racist errors

#50

Post by changoplatanero »

rjh29 wrote: post #49
changoplatanero wrote: post #45

Earlier quoted context omitted.

But, white people get misclassified as animals by the classifier too. Typically white people aren't misclassified as gorillas but as other animals. So i don't think the cause is as simple as non-white people not being considered during training.

It classified 80 photos of the same black person as 'gorilla', I have not heard of that happening with white people.

I saw lots of examples of white children being classified as seals
Post reply on HN