Approaching fairness in machine learning • Hacker News Board

Re: Approaching fairness in machine learning

#21

Post by skybrian » Wed, Sep 07, 2016, 9:17 PM UTC

yummyfajitas wrote: post #14

danso wrote: post #12

Earlier quoted context omitted.

Yes, blacks are fundamentally different from whites in terms of the available data to train algorithms on: http://www.nytimes.com/2015/10/31/nyregion/hudson-city-bank-... > The government’s analysis of the bank’s lending data shows that Hudson’s competitors generated nearly three times as many home loan applications from predominantly black and Hispanic communities as Hudson did in a region that includes New York Cit…

Did you read what I wrote? Available data on blacks specifically is completely irrelevant if blacks and whites aren't fundamentally different. The white model will generalize. If repayment probability for blacks and whites alike is is A x downpayment_fraction + B x credit_score, you can use training data from whites and the model will accurately predict black repayment probability. It only fails if you actually need…

I don't know what you mean by "fundamentally different" but there are definitely going to be demographic differences that the algorithm could use to predict race with good probability from hidden variables. (Where they live, for example.) History has an influence that's hard to remove from the dataset.

I'd guess that another reason this problem is hard is that it's about defining the goal correctly. It's not just maximizing repayment. There is some fairness goal that isn't well-defined.

Re: Approaching fairness in machine learning

#22

Post by closed » Wed, Sep 07, 2016, 9:19 PM UTC

yummyfajitas wrote: post #14

danso wrote: post #12

Earlier quoted context omitted.

Yes, blacks are fundamentally different from whites in terms of the available data to train algorithms on: http://www.nytimes.com/2015/10/31/nyregion/hudson-city-bank-... > The government’s analysis of the bank’s lending data shows that Hudson’s competitors generated nearly three times as many home loan applications from predominantly black and Hispanic communities as Hudson did in a region that includes New York Cit…

Did you read what I wrote? Available data on blacks specifically is completely irrelevant if blacks and whites aren't fundamentally different. The white model will generalize. If repayment probability for blacks and whites alike is is A x downpayment_fraction + B x credit_score, you can use training data from whites and the model will accurately predict black repayment probability. It only fails if you actually need…

Even in models where race doesn't directly cause an outcome, a model's judgements may be biased against a race.

For example, suppose that (1) people can be green or blue, (2) green people tend to live in Idaho, (3) living in Idaho is associated with people not paying back loans.

A linear model where there are only non-zero, positive coefficients for the path p(green) -> p(Idaho) -> p(fail_to_repay), and p(credit_score) -> p(fail_to_repay) will create trouble, even though color does not directly affect repayment. If you use a multiple regression with fail_to_repay ~ B0 + B1Idaho + B2credit_score, it will discriminate against green people, by penalizing people from Idaho.

AFAIK, one of the points of the paper linked in the parent comment is that blindly using indicators like IP address may indirectly lead to discrimination against a racial group in this way, e.g. p(racial_group) -> p(a_specific_IP_address).

Maybe more relevant to your example, though, is that assuming whites and blacks have the same model in the "ground-truth" scenario I presented could cause a model to be discriminative (when it shouldn't be, because the coefficient for the path from p(green) -> p(fail_to_repay) is 0).

This specific issue is hairy, and exists for traditional approaches also.

Re: Approaching fairness in machine learning

#23

Post by closed » Wed, Sep 07, 2016, 9:23 PM UTC

danso wrote: post #19

yummyfajitas wrote: post #18

Earlier quoted context omitted.

Ok, so now we have all acknowledged that we are "race realists" or "scientific racists" in this conversation. ( https://en.wikipedia.org/wiki/Scientific_racism ) Anyway we've now accepted blacks and whites may behave differently. For example, lets suppose we have all the training data we need to accurately recognize that one race doesn't pay back their loans as much as others, all else held equal. What should we do a…

I don't know if I nor Dr. King Jr. have to subscribe to scientific racism just because we subscribe to the reality that folks with of different racial backgrounds have a higher probability of being shortchanged historically. And thus, that any machine learning approach that doesn't factor this in will risk perpetuating such disadvantages, which kind of defeats the ostensible purpose for using machine learning to appl…

FWIW, I present a case where race does not directly cause increased failure to repay, but common approaches to modeling could discriminate against race.

These issues have been discussed in detail in statistical considerations of Simpson's paradox. One need not accept that racial differences directly affect an outcome of interest, in order to be concerned about a model being biased against race!

Re: Approaching fairness in machine learning

#24

Post by skybrian » Wed, Sep 07, 2016, 9:23 PM UTC

eanzenberg wrote: post #11

tlb wrote: post #5

Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf . It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will…

If this is the case competition will weed it out.

That's... optimistic. In the long run, maybe, but someone has to actually do it.

Re: Approaching fairness in machine learning

#25

Post by bo1024 » Wed, Sep 07, 2016, 9:24 PM UTC

Eridrus wrote: post #20

The biggest issues of bias/fairness in ML are not to do with the algorithms or results, but the underlying data. A trivial example would be: what if you trained a classifier to predict whether a person would be re-arrested before they went to trial? Some communities are policed more heavily so you would tend towards reinforcing the bias that exists and provide more ammunition to those arguing for further bias in the…

Great points. These are exactly the kinds of issues researchers are grappling with in trying to nicely define, and attempt to achieve, fairness in ML.

Re: Approaching fairness in machine learning

#26

Post by [deleted] » Wed, Sep 07, 2016, 9:32 PM UTC

tlb wrote: post #5

Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf . It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will…

[deleted]

Re: Approaching fairness in machine learning

#27

Post by yaireli » Wed, Sep 07, 2016, 9:41 PM UTC

tlb wrote: post #5

Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf . It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will…

Well, estimating higher risk due to lack of information, is not a glitch - rather the rational correct estimation. Say you're a complete stranger and want to hangout with me, this is pretty scary! However, if I know you, and you're a jerk - you might piss me off during the night, but at least I know you're not a serial killer...

Re: Approaching fairness in machine learning

#28

Post by carapace » Wed, Sep 07, 2016, 9:43 PM UTC

fatdog wrote: post #6

What is fairness but political accountability? There is an old joke about how people use statistics like a drunk uses a lamp post: for support and not for illumination. Given this, we can expect people to use AI like everything else in statistics, to support the agenda of whoever is operating it while defraying negative personal accountability for the results, because artificial intelligence. It's just an obfuscated…

I wish I could upvote this remark twice (or more.)

This is pretty much the only important concept for figuring out how we will use this tech politically.

Because it takes genius-level intelligence to be able to figure out whether you're just telling yourself what you want to hear, and incredibly rare responsibility to remember to [keep on trying to] do so, individuals and tiny groups may be able to use AI for these sorts of things, but large groups, municipalities, states, corps, etc. never will.

The systems we can understand and manage as a group are vastly simpler than those which you can understand and manage as an individual.

Re: Approaching fairness in machine learning

#29

Post by yummyfajitas » Wed, Sep 07, 2016, 9:46 PM UTC

closed wrote: post #22

yummyfajitas wrote: post #14

Earlier quoted context omitted.

Did you read what I wrote? Available data on blacks specifically is completely irrelevant if blacks and whites aren't fundamentally different. The white model will generalize. If repayment probability for blacks and whites alike is is A x downpayment_fraction + B x credit_score, you can use training data from whites and the model will accurately predict black repayment probability. It only fails if you actually need…

Even in models where race doesn't directly cause an outcome, a model's judgements may be biased against a race. For example, suppose that (1) people can be green or blue, (2) green people tend to live in Idaho, (3) living in Idaho is associated with people not paying back loans. A linear model where there are only non-zero, positive coefficients for the path p(green) -> p(Idaho) -> p(fail_to_repay), and p(credit_scor…

If I understand your model right, you are saying that Idahoans don't repay loans and your model accurately reflects this. This isn't a bias at all. The model is issuing fewer loans to green people not because they are green but because they live in Idaho and are unlikely to pay back said loans.

This is a case like what is described in the article - when a perfect predictor (another word for this is "reality" or "hindsight") will still exhibit disparate impact.

Re: Approaching fairness in machine learning

#30

Post by yummyfajitas » Wed, Sep 07, 2016, 9:58 PM UTC

skybrian wrote: post #21

yummyfajitas wrote: post #14

Earlier quoted context omitted.

Did you read what I wrote? Available data on blacks specifically is completely irrelevant if blacks and whites aren't fundamentally different. The white model will generalize. If repayment probability for blacks and whites alike is is A x downpayment_fraction + B x credit_score, you can use training data from whites and the model will accurately predict black repayment probability. It only fails if you actually need…

I don't know what you mean by "fundamentally different" but there are definitely going to be demographic differences that the algorithm could use to predict race with good probability from hidden variables. (Where they live, for example.) History has an influence that's hard to remove from the dataset. I'd guess that another reason this problem is hard is that it's about defining the goal correctly. It's not just max…

By "fundamentally different", I mean that the most accurate model will be something like this:

    repayment_probability = 1 x downpayment_frac + 0.5 x credit_score + A x isBlack

for some A != 0. I.e., if A = -0.2, then a black borrower with a 60% downpayment is as likely to pay back a loan as a white borrower with a 40% downpayment.

If A = 0, then the bias described by tlb and danso won't occur.

What you describe with hidden variables is called "redundant encoding", and it's just a way of recovering the `A x isBlack` term if you remove `isBlack` from your input set. But if blacks and whites repay their loans at the same rate (holding all else equal), redundant encoding won't happen - it doesn't actually improve accuracy.

I describe this in more detail here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

I agree with you that the core issue is an unspecified true goal. Folks are unwilling to publicly and explicitly state how many bad loans should be issued for fairness or how many unqualified students should be allowed into college for diversity.

Or for an example closer to home, how much we should lower the bar in order to hire more non-Asian minorities in tech? Daring to ask that question gets you some pretty hostile responses.