Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on AI Bias

ben-evans.com

Post reply on HN
Print view View original

91–100 of 126 posts

Re: Notes on AI Bias

#91

Post by wongarsu »

meheleventyone wrote: post #73
DuskStar wrote: post #57

Earlier quoted context omitted.

Of course, sometimes reality is also biased, and the AI systems are just accurately reflecting reality. And that's an even bigger elephant.

I’m not sure what that even means if we know we can bias outcomes. Pretending there is some kind of natural state that is for the sake of being natural preferred seems odd given humans propensity to change the world to suit. I also suspect for many that ‘reality’ is really just a dog whistle for their preferred biases. Not to mention the entire issue with deriving and ought from an is.

Suppose you train an AI to predict how good people are at weight lifting, trained from a bunch of seemingly unrelated data (maybe you want to hire bouncers or construction workers). You will find that the model predicts better performance for males. You notice this, identify that men are more likely to go to the gym than wimen, and modify your data to compensate for this. But when you rerun the model men still show better results. You find some other biases in your data. You find societal biases, like role models for girls not being physically strong. You even take some women and show that with training they outperform average men.

You can modify reality, but our understanding of biology - especially hormones - clearly tells us that the AI was right: men are generally better than women at weight lifting.

I'm not saying that every issue is like that, but it would be foolish to ignore that sometimes reality is biased, sometimes in obvious ways and sometimes more subtly.

Re: Notes on AI Bias

#92

Post by theoh »

cygaril wrote: post #75
theoh wrote: post #7

Earlier quoted context omitted.

Consider the possibility that the (pre-AI system) probability of success for a female applicant is the same as the probability of success of a male applicant. You could make a "per capita" quota as a kind of goal. That's not a problem, but how would you make sure the quota was met? The typical AI system doesn't work on the basis of selecting candidates entirely at random, pro rata, in order to meet a quota. It works…

From a machine learning point of view, one can just add the constraint that the probability of being in the "yes" bucket is that same for both male and female candidates. Doing this will give a worse fit than an unconstrained optimization, but it is fairer. More sophisticated approaches are possible.

There's no "just" to any aspect of this topic. I think what you are talking about is what is sometimes called "classification parity", and there are problems with it, and with everything else we've come up with to combat bias.

https://arxiv.org/abs/1808.00023

Re: Notes on AI Bias

#93

Post by wongarsu »

Someone wrote: post #40

Short way to describe the problem: we want to build systems that detect causation, but statistical models can only detect correlation.

That's not entirely true: it's hard to show causation, but with enough data you can. If A correlates with B you know that either A causes B, B causes A, some C causes both A and B, or the correlation is a coincidence. If you have the data to rule out 3 of those the remaining possibility is the causation.

Re: Notes on AI Bias

#94

Post by jakelazaroff »

FakeComments wrote: post #66
michaelmior wrote: post #32

Earlier quoted context omitted.

Race and culture are two very different things. There certainly can be some correlation, but drawing conclusions of suitability for employment based on race is a problem.

I agree — race and culture are very different things. That’s why it’s only meaningful to draw these conclusions once you control for cultural variation. No one has shown that they drew conclusions based on race: they drew conclusions based on written words, which correlated with race. The claim is that this demonstrates racial discrimination. However, an equally valid explanation is culture as a confounding variable…

"X correlates with race" and "X correlates with culture, which correlates with race" are the same thing. If neither X nor culture are related to what you're trying to measure, it's still just racial bias.

We've been using "ice hockey" as an example, right? Given that it's not obviously correlated with fitness for a job at Amazon, it's incumbent on you to show that it actually is. Until then, we should assume that it's at best a false positive from unrelated data (like the ruler in the skin cancer example) and at worst a proxy for race and gender.

Re: Notes on AI Bias

#95

Post by gambler »

joshuamorton wrote: post #77
gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

>Framing this problem as "bias" Except that's exactly what it is. Much as your model was biased against interns. > and especially hyper-focusing everyone's attention on diversity aspect of it is extremely irresponsible. Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? Just because the same kind of methodological flaw can cause other harms its irresponsible to use a motiv…

>Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible?

In my opinion, yes, if it leads most readers to misjudge some fundamental properties of the problem as a whole. Again, I'm not saying this article is guilty, but most are.

Re: Notes on AI Bias

#96

Post by bumby »

BickNowstrom wrote: post #87
bumby wrote: post #85

Earlier quoted context omitted.

Isn't this the one of the major concerns of ML, the bias-variance trade-off? By creating a low-variance model, we create a highly biased model that misses some of the important feature relationships necessary to create a truly generalized model? Meaning, isn't it prudent to spending time on this issue?

You are conflating bias (error) with bias (fairness).

Haha my comment was originally drafted to accuse the parent comment of the same. As I read the article, its concerned with error (e.g. misclassification of cancer) but the parent comment translated this to mean the social bias.

Re: Notes on AI Bias

#97

Post by joshuamorton »

gambler wrote: post #95
joshuamorton wrote: post #77

Earlier quoted context omitted.

>Framing this problem as "bias" Except that's exactly what it is. Much as your model was biased against interns. > and especially hyper-focusing everyone's attention on diversity aspect of it is extremely irresponsible. Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? Just because the same kind of methodological flaw can cause other harms its irresponsible to use a motiv…

>Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? In my opinion, yes, if it leads most readers to misjudge some fundamental properties of the problem as a whole. Again, I'm not saying this article is guilty, but most are.

> In my opinion, yes, if it leads most readers to misjudge some fundamental properties of the problem as a whole.

Which problem? The general statement of this problem is "models, trained on [somehow] misrepresentative data [or even technically representative data] can draw unintended conclusions that lead to harm". Specifically in this case, the harm was "the model was basically just trained to ignore all women applicants due to bad inference of conditional probabilities".

This is a common thing. Because our society draws lines and has bias, its fairly common for modelling failures to exist along those lines. Indeed, sometimes the failures are mostly harmless and immediately obvious, but often they aren't. And people building models should be made aware of those failure scenarios, and be especially aware of failure scenarios that affect underrepresented groups, because those groups are the most likely for the model to fail on if you aren't actively looking for them.

And this stuff is pervasive. Facial recognition tech is much worse at noticing the faces of darker skinned people [1]. Some of this is because the people building the common models (eigenfaces etc.) didn't use diverse skin tones, but some of it goes back further, white balance in film was tuned for lighter skin tones until the 90s[2]. Some of that has likely persisted into modern film and camera technology, unfortunately. People working with data need to understand their data. And that means understanding how bias infests their data.

> fundamental properties of the problem as a whole

You've yet to state the "whole problem" or the fundamental properties that people might misjudge. So I'm unclear what they are.

[1]: Arguably an advantage now.

[2]: https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f...

Re: Notes on AI Bias

#98

Post by Someone »

wongarsu wrote: post #93
Someone wrote: post #40

Short way to describe the problem: we want to build systems that detect causation, but statistical models can only detect correlation.

That's not entirely true: it's hard to show causation, but with enough data you can. If A correlates with B you know that either A causes B, B causes A, some C causes both A and B, or the correlation is a coincidence. If you have the data to rule out 3 of those the remaining possibility is the causation.

So, how do you, for example, rule out “some C causes both A and B“, if you may not even know of the existence of C?

More importantly, the only way to really show causation is by positing a mechanism.

Re: Notes on AI Bias

#99

Post by btilly »

DuskStar wrote: post #21
fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

One situation I could see leading to this result (Amazon cancelling their resume filtering software with the excuse that it 'skewed male') is that 1. The AI system accurately predicted employee success across both genders AND 2. The AI system predicted that women would do worse than men That's politically embarrassing and something that you can't necessarily 'fix' by improving the system. (see: all the 'will this per…

One major problem.

The parole software was NOT being fed data for "will this person commit another crime". It was being fed data for, "will this person be a suspect for another crime".

The significant difference is that selective enforcement biases the data that it was trained on. Said selective enforcement has multiple causes, including the fact that heavier patrolling in black neighborhoods makes catching crimes more likely.

The size of the selective enforcement bias shows in a number of ways. For example consider drugs. In surveys, the usage of illegal drugs is the same in blacks and whites. And yet 6 times as many blacks are arrested for using illegal drugs as whites.

Re: Notes on AI Bias

#100

Post by [deleted] »

twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

[deleted]
Post reply on HN