Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

81–90 of 201 posts

Re: Attacking discrimination with smarter machine learning

#81

Post by SerLava »

tomp wrote: post #49
Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

>if you took into account enough circumstances (e.g. single parent, school district, income level, parents' wealth) you'd be able to remove race from your model and still arrive to the "equal opportunity" result. That way, you wouldn't discriminate based on race, but you would still help the people most needy of help (poor, uneducated, etc.).

This is pretty much the only way to help people who were born addicted to meth inside a trailer in the mountains who also happen to be blessed with being white.

Re: Attacking discrimination with smarter machine learning

#82

Post by BickNowstrom »

tptacek wrote: post #24
mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

It is possible to inspect black box models. See for instance http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf and https://homes.cs.washington.edu/~marcotcr/blog/lime/ . It is also common to use highly accurate white box models for cases where it is important to not overfit, leak, or discriminate, like MARS and GA^2M (somewhat out of scope for an actuary using R).

If a computer program spots an irrelevant unhelpful correlation its generalization error will noticeable go up. If it is an irrelevant "helpful" correlation it means there is a problem with the data (such as leakage), not with the algorithm. If there is a problem with the data, all bets are of, both for black and white box models.

A blackbox model will probably not find that being an Armenian alone will lead to more crashes. Being non-linear in nature it will find interactions (young male Armenians are more likely to crash than young males in general). If the importance of such a feature is not significant enough to distinguish it from noise, then regularization may automatically remove it.

Even if we observe that 99 out 100 Armenians crash their cars, and you decline someone a loan, because he/she is Armenian, you may just have discriminated against the 1 Armenian who is a safe driver. Young male drivers who drive safely have a worse time getting loans, because their group (the set of young male drivers) spoiled it for them. So their only hope of getting a loan is you adding more features (like nationality), to be able to distinguish them as safe drivers, not removing them and lumping them into the status quo.

Re: Attacking discrimination with smarter machine learning

#83

Post by gaius »

neolefty wrote: post #77
NotSureYet wrote: post #55

Earlier quoted context omitted.

It's OK because we subsidize their Health Care: men and women pay the same for the same age, but women use way more than men.

Citation?

Here you go: https://www.ncbi.nlm.nih.gov/pubmed/18439060

Re: Attacking discrimination with smarter machine learning

#84

Post by weberc2 »

Retric wrote: post #78
weberc2 wrote: post #66

I agree that this should be a possibility, while I think the odds that millions of law enforcement officers and criminal justice faculty would spontaneously conspire against a particular race, I also think it's more likely than some genetic predisposition toward crime (i.e., the first option).

Now, this is his first offence, but a young man is found high on PCB walking down the street hitting cars with a baseball bat. It takes five cops and a significant struggle to arrest him. Having read that, what would you assume is race and economic background is? After criminal proceedings he was sentenced to community service. Now, what would you assume is race and economic background is? PS: Bias is insidious and r…

I wouldn't make assumptions, but I don't understand your point.

Re: Attacking discrimination with smarter machine learning

#85

Post by FT_intern »

tomp wrote: post #49
Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

(2) is not completely true because some impoverished communities are more prone to commit crimes than others.

Biologically, (1) is not true. Culturally, it may be true. Impoverished black people idolize and preach a culture that encourages crimes and the precursors that make criminal actions more likely.

Re: Attacking discrimination with smarter machine learning

#86

Post by sangnoir »

zmoreira wrote: post #67
corey_moncure wrote: post #61

Earlier quoted context omitted.

> The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: I'd like to add a third possible reason for your consideration. Since "criminality" i.e. guilt of committing a crime is determined after a process engaging the law enforcement and justice systems, we have to examine whether there are inherent biases in those systems that result in ske…

This is studiable and has been studied. A study regarding police killings, for example: Do White Police Officers Unfairly Target Black Suspects? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2870189&... Using a unique data set we link the race of police officers who kill suspects with the race of those who are killed across the United States. We have data on a total of 2,699 fatal police killings for the years…

> This is studiable and has been studied

True, but there has been more than one paper written on the subject, which don't all agree with the one you linked vis over-representation for crimes.

The black/white marijuana arrest gap, in nine charts

https://www.washingtonpost.com/news/wonk/wp/2013/06/04/the-b...

As you're probably aware, black Americans are arrested for marijuana possession far more frequently than whites. You may also know that there's not much evidence that black people consume marijuana with greater regularity than whites do.

...And this is a uniform phenomenon. It's not that some states treat the races equally and others treat them really unequally. Only in Hawaii are the rates even close to equal, and that's biased by the fact that blacks make up only 1.6 percent of the population. In the state with the second-lowest disparity, Alaska, blacks are 1.6 times more likely to be arrested. In the state with the biggest, Iowa, blacks are 8.34 times more likely to be arrested. D.C. has the second biggest; in the District, blacks are 8.05 times more likely to be arrested.

Re: Attacking discrimination with smarter machine learning

#87

Post by [deleted] »

deong wrote: post #75
Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

But how do you do that? Race is baked into a lot of the attributes your classifier is going to find informative. Is someone a good credit risk? To decide, you look at features like past payment history, available balance, zip code, etc. Past payment history: if you're black, prior discriminatory behaviors may have limited your ability to open credit accounts, and thus you have less history to go on. Available balance…

[deleted]

Re: Attacking discrimination with smarter machine learning

#88

Post by icebraining »

d--b wrote: post #8

Nice to see that the debate has reached the ears of the main people working on this field. What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve. In other words, the objective function of the optimization problem needs not only match the current state of the world, and provide an hindsight in one's own economic interests, it also needs to take into account th…

I'm not sure that's clear. There are actually two ways to achieve the outcome we want; tweaking the model or changing the inputs.

What I mean, say the model identifies that a certain group has a greater risk due to systemic problems. If you change something about the group, you can change the calculated risk without changing the model. And this may very well be a better way to achieve the outcome you want.

Specifically, by preventing insurance companies from using a more accurate mode, what you're demanding is that the random people who happen to have taken the same insurance packages but are not part of the group should make an extra contribution to fix these systemic problems.

But why them? Shouldn't we all contribute instead, hopefully using a fair system for assessing how much should each pay?

Instead of tweaking the model, you can change the inputs by providing a state-backed guarantee to the underprivileged groups. Isn't it more fair overall?

Re: Attacking discrimination with smarter machine learning

#89

Post by d--b »

alvarosm wrote: post #46

You realize you're advocating totalitarianism, right?

Race and ability to repay a loan, are - and this is scientifically proven - uncorrelated. What is correlated is your social status and your ability to repay a loan.

However, there is a problem because the population of the US is not uniform. Because of history, some races were less wealthy than others. So far, this makes sense I suppose.

Now, talking about machine learning. The big specificity of machine learning (when it is implemented with neural nets) is that it is trained on data only. The machine doesn't know about biology, and it doesn't know about history. Therefore, the model sort of converges to a place where effectively it "believes" from the data it has seen that there is a correlation between race and ability to repay a loan. So basically, the historical bias that is in the data translates into a logical bias. And so, because of history, you're less likely to get a loan if you're a black person than if you're a white person. That is what I call "amplifying the status quo"

Now, as we live in a society that holds equality of chances as one of its core values, I think it is good that people who work in machine learning look at these bias, and have a criticism towards the results of the algorithms in terms of societal impacts, and not only in terms of economical gain.

This has nothing to do with totalitarianism. Yes it is not libertarian, but we don't live in a libertarian world.

Re: Attacking discrimination with smarter machine learning

#90

Post by mspeedy »

tomp wrote: post #49
Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

> if you took into account enough circumstances (e.g. single parent, school district, income level, parents' wealth) you'd be able to remove race from your model and still arrive to the "equal opportunity" result.

The problem is, when given access to a large number of classifiers, some of which have inevitably been affected by a pre-existing racial bias, a black box machine learning algorithm will likely become discriminatory as well if race is not in some way represented and equalized.

For instance, many justice systems in the U.S. use machine learning software to determine the likelihood that a criminal will reoffend, and use that prediction to determine sentencing. Race is never used explicitly as a classifier, but the program ended up being significantly more likely to rate blacks as more likely to reoffend [1]. Classifiers like "had parents with previous criminal convictions" can be misleading when blacks are more likely to be convicted for the same crime as whites. It doesn't mean that the white person's parents didn't engage in criminal activity or other reprehensible behavior that might cause their child to become a violent, repeat offending criminal - just that they were able to get away with it more easily because of a biased system.

Machines end up just as biased as the data they've been trained on, so if we are going to use computers to judge things that have such a significant impact on people's lives, we can't risk racism slipping through the cracks.

[1] https://www.propublica.org/article/machine-bias-risk-assessm...

Post reply on HN