Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Approaching fairness in machine learning

blog.mrtz.org

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Approaching fairness in machine learning

#31

Post by yummyfajitas »

danso wrote: post #19
yummyfajitas wrote: post #18

Earlier quoted context omitted.

Ok, so now we have all acknowledged that we are "race realists" or "scientific racists" in this conversation. ( https://en.wikipedia.org/wiki/Scientific_racism ) Anyway we've now accepted blacks and whites may behave differently. For example, lets suppose we have all the training data we need to accurately recognize that one race doesn't pay back their loans as much as others, all else held equal. What should we do a…

I don't know if I nor Dr. King Jr. have to subscribe to scientific racism just because we subscribe to the reality that folks with of different racial backgrounds have a higher probability of being shortchanged historically. And thus, that any machine learning approach that doesn't factor this in will risk perpetuating such disadvantages, which kind of defeats the ostensible purpose for using machine learning to appl…

Historically isn't the issue. The issue is a simple factual question of whether, all else held equal, black people repay their loans at the same rate as whites in identical financial circumstances. The fact that in aggregate financial circumstances might be different isn't important to this question.

If they do, then you don't need to worry about algorithms discriminating. Insofar as they do it's merely a sampling error (i.e. shrinks like O(1/sqrt(N)), where N = Nwhite + Nblack) and they are just as likely to discriminate in favor as against.

If they don't, then you subscribe to scientific racism, or the belief that blacks and whites in identical circumstances behave fundamentally differently.

(I describe these different cases in explicit detail here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... )

So do you believe race affects reality independent of other factors? And assuming you do subscribe to scientific racism, what should we do about it?

Re: Approaching fairness in machine learning

#32

Post by throw_away_777 »

wyager wrote: post #13

Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…

This is a great point. People believe that most groups are basically equal; this is true in the sense that if people were raised in identical environments with equal opportunities than it probably wouldn't really matter what group they were in, but wrong because that isn't the world we live in. Different groups on average experience much different environments. Machine learning doesn't care why the differences in groups arises, but people do. Fundamentally the question is whether we want to base our decisions based on how the world is, or on how we want the world to be.

Re: Approaching fairness in machine learning

#33

Post by danso »

yummyfajitas wrote: post #31
danso wrote: post #19

Earlier quoted context omitted.

I don't know if I nor Dr. King Jr. have to subscribe to scientific racism just because we subscribe to the reality that folks with of different racial backgrounds have a higher probability of being shortchanged historically. And thus, that any machine learning approach that doesn't factor this in will risk perpetuating such disadvantages, which kind of defeats the ostensible purpose for using machine learning to appl…

Historically isn't the issue. The issue is a simple factual question of whether, all else held equal, black people repay their loans at the same rate as whites in identical financial circumstances. The fact that in aggregate financial circumstances might be different isn't important to this question. If they do, then you don't need to worry about algorithms discriminating. Insofar as they do it's merely a sampling er…

> The issue is a simple factual question of whether, all else held equal, black people repay their loans at the same rate as whites in identical financial circumstances

Oh if you put it that way, then I don't know. Because that's not the reality that's being dealt with, in which whites and blacks have identical circumstances. I think you're reading something into this that others aren't.

Re: Approaching fairness in machine learning

#34

Post by tlb »

yummyfajitas wrote: post #7
tlb wrote: post #5

Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf . It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will…

Of course, this theory of discrimination is only applicable when minorities are fundamentally different from majorities . I.e., if the same ruleset is accurate for both whites and blacks (i.e., "I don't care about race, if he puts 20% down he's good"), this argument doesn't work at all - you can train your model on everyone and it'll work just fine. However, if blacks and whites need to be treated fundamentally diffe…

In that case of being treated differently (requiring a different amount of downpayment as security), it's probably racist. More common and less controversial is the case when the signals are in different channels.

For instance, when dealing with immigrants, US banks often fail to see any signal at all because their credit reporting only covers US institutions, and they don't know how to verify employment or schooling abroad. So to start making loans to immigrants from any given country, they need to figure out what the signals are (job, schooling, ...) and how they correlate with risk.

Re: Approaching fairness in machine learning

#35

Post by closed »

yummyfajitas wrote: post #29
closed wrote: post #22

Earlier quoted context omitted.

Even in models where race doesn't directly cause an outcome, a model's judgements may be biased against a race. For example, suppose that (1) people can be green or blue, (2) green people tend to live in Idaho, (3) living in Idaho is associated with people not paying back loans. A linear model where there are only non-zero, positive coefficients for the path p(green) -> p(Idaho) -> p(fail_to_repay), and p(credit_scor…

If I understand your model right, you are saying that Idahoans don't repay loans and your model accurately reflects this. This isn't a bias at all. The model is issuing fewer loans to green people not because they are green but because they live in Idaho and are unlikely to pay back said loans. This is a case like what is described in the article - when a perfect predictor (another word for this is "reality" or "hind…

It is a bias if you calculate the cost to people taking out loans, based on color. Green people will pay a higher cost, even though in the ground-truth model their race is not directly related to loan repayment.

For example, if only blue people in Idaho fail to repay loans, green people will still absorb a greater cost in the multiple regression case above (in the sense that they are more likely to be penalized for being Idahoans).

Re: Approaching fairness in machine learning

#36

Post by contravariant »

wyager wrote: post #13

Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…

I think you're right and obviously making the machine make suboptimal decisions is definitely not a good solution.

However I think a case can be made that certain protected attributes should be censored. Not to prevent the algorithm from making optimal decisions, but to prevent it from overfitting on those attributes. Which, if you think about it, is essentially what discrimination is.

Re: Approaching fairness in machine learning

#37

Post by rubyfan »

I am actually optimistic on Big Data's effect in equality.

Small data is actually kind of the problem. When you have limited ability to process data or limited data density then your segmentation ability is limited to small data like state, county, zip code, credit score, whether you own a home, etc.

Big data processing, big bad ML algorithms and the ubiquity of data is making advanced segmentation available that allows us to make arguably more equitable outcomes.

Re: Approaching fairness in machine learning

#38

Post by h4nkoslo »

yummyfajitas wrote: post #2

After studying this issue, and learning a lot more about learning and optimization, I've come to the conclusion that the best solution [1] is probably explicit racial/sexual/other special interest group quotas. Specifically, we should train a classifier on non-Asian minorities. We should train a different classifier on everyone else. Then we should fill our quotas from the non-Asian minority pool and draw from the pr…

The problem is that you run out of "good" NAMs (or women with the exact same career preferences as men, etc.) extremely quickly. The demand for "good" NAMs in any given field vastly exceeds supply, since quotas tend to be set at population proportion.

Re: Approaching fairness in machine learning

#39

Post by h4nkoslo »

Eridrus wrote: post #20

The biggest issues of bias/fairness in ML are not to do with the algorithms or results, but the underlying data. A trivial example would be: what if you trained a classifier to predict whether a person would be re-arrested before they went to trial? Some communities are policed more heavily so you would tend towards reinforcing the bias that exists and provide more ammunition to those arguing for further bias in the…

These are not "fairness" issues; these are process feedback issues. The same problem pops up if you're using algorithmic selection of machine parts to test for failure, attempting to programmatically evaluate patches for code quality, writing fraud detection algorithms, etc.

Re: Approaching fairness in machine learning

#40

Post by yummyfajitas »

danso wrote: post #33
yummyfajitas wrote: post #31

Earlier quoted context omitted.

Historically isn't the issue. The issue is a simple factual question of whether, all else held equal, black people repay their loans at the same rate as whites in identical financial circumstances. The fact that in aggregate financial circumstances might be different isn't important to this question. If they do, then you don't need to worry about algorithms discriminating. Insofar as they do it's merely a sampling er…

> The issue is a simple factual question of whether, all else held equal, black people repay their loans at the same rate as whites in identical financial circumstances Oh if you put it that way, then I don't know. Because that's not the reality that's being dealt with, in which whites and blacks have identical circumstances. I think you're reading something into this that others aren't.

Because that's not the reality that's being dealt with, in which whites and blacks have identical circumstances.

Of course it is. There may be 5 blacks and 100 whites with a credit score of 830. But as long as blacks and whites with an 830 credit score behave the same, then data from whites will generalize to blacks and the problem tlb brought up doesn't apply. Redundant encoding is also irrelevant - this is useless information so an accuracy maximizer has no reason to pay any attention.

Insofar as blacks and whites with an 830 credit score behave differently, then algorithms might treat them differently. That's the "race realism" hypothesis.

Post reply on HN