Google Photos AI still can't label gorillas after racist errors

> I wonder if they're over-reacting

There doesn't seem to be any benefit to solving this problem, so how is it an overreaction given the negative press?

> At the same time, I'm surprised they can't get this classifier to work. It doesn't seem like a very challenging problem

Why are you assuming that they can't get the classifier to work? There's no evidence to that effect in the article.

> I wonder if they're over-reacting and just deciding to zero any residual risk by not allowing that classification anymore.

Why do you describe that as over-reacting? It seems like the appropriate level of reaction given there is no upside for getting it right and a massive downside for getting it wrong.

It doesn't even need to be a top-down edict, it's just the way the incentives will work out for everyone involved.

Let's say that you're an engineer working on that team, and find that there's a bunch of terms added to a blocklist a decade ago and still there. Are you going to just remove them, on the assumption that the classifications are correct? Of course not. How much validation work are you willing to do to convince yourself and others that the results are 100% correct? Is that really going to be the most interesting or impactful work you can do this quarter?

Similarly you'd find that whoever needs to approve such a removal would have incentives very strongly biased toward not approving the removal of terms from a blocklist. If everything goes well, they get no credit. If something goes wrong, they get the blame. It's possible to push through that bias toward inaction, but it requires it to be a change that somebody really wants to make happen.

I read the entire wikipedia page on racism to see why people call a bug in an experimental product feature, based on bad classifiers and data, a form of racism.

It's unfortunate. And I can see why it worries people. Because on the surface it looks like a systemic problem. But is it really?

I think I could write a classifier that would accurately separate out gorillas and humans in a weekend with a dataset. But I guess as its pointed out here, there's no upside to getting it right and massive downside to getting it wrong

I think we can all agree that the classifier is horribly broken.

But it seems like if nobody is working on this, how will we ever fix this gaping hole in image classifiers? And don't we want to fix it? And to fix it, research will continue to get it wrong until they get it less wrong and more right, but can only iterate without a massive backlash. It seems like being stuck between a rock and a hard place.

I am rhetorically asking, wouldn't we have to allow researchers to iterate on this problem to fix it? That simply won't happen until we are able to allow them leeway understanding that this is an incrementally improving model. Otherwise what we have is just a sledgehammer solution (just banning all primate classifications) which actually never addressed the problem, that these models do have a race-based bias (probably in their input datasets.)