[deleted]
Approaching fairness in machine learning
41–50 of 56 posts
Re: Approaching fairness in machine learning
#42Earlier quoted context omitted.
If I understand your model right, you are saying that Idahoans don't repay loans and your model accurately reflects this. This isn't a bias at all. The model is issuing fewer loans to green people not because they are green but because they live in Idaho and are unlikely to pay back said loans. This is a case like what is described in the article - when a perfect predictor (another word for this is "reality" or "hind…
It is a bias if you calculate the cost to people taking out loans, based on color. Green people will pay a higher cost, even though in the ground-truth model their race is not directly related to loan repayment. For example, if only blue people in Idaho fail to repay loans, green people will still absorb a greater cost in the multiple regression case above (in the sense that they are more likely to be penalized for b…
Yes, if it's actually (blue & Idaho) ~> default, and your model ignores blue, then the greens will pay a higher cost. If color is redundantly encoded then your model can partially fix this and penalize the blue's in Idaho.
Do you consider this situation unjust? If so, you might be unhappy to learn that the entire goal of the field of algorithmic fairness is to do something along these lines.
Re: Approaching fairness in machine learning
#43Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf . It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will…
Of course, this theory of discrimination is only applicable when minorities are fundamentally different from majorities . I.e., if the same ruleset is accurate for both whites and blacks (i.e., "I don't care about race, if he puts 20% down he's good"), this argument doesn't work at all - you can train your model on everyone and it'll work just fine. However, if blacks and whites need to be treated fundamentally diffe…
Lives of members of social groups can be different for historical reasons. Because of this, the best selection of features (as in, how we select and encode relevant aspects of a dataset for a particular problem) that one would need to use, as well as the correlations between them, may be different between different groups. The question is not whether there is fundamentally a difference between, in this case, racial groups vis-a-vis paying back loans (i.e. that the only feature required in your model would be "is member of this group"), but what are the traits of possibilities of life that have for whatever reason ended up leaving a quantifiable trace in our databases and their distribution within that group.
One hypothetical example: suppose that there existed a group G that was not able to go to the top n% of universities due to discrimination. Your company uses some rank of university attended as one of the features input to its favorite machine learning algorithm. However, the dataset you trained on excluded group G. Within this group, the best university individuals have been able to attend is X which is by definition not in the top n%. Had the algorithm been trained on this group it would have observed that school X is highly correlated with success in this group, even if not in the original training set used. As is, your ML system assigns a low probability to members of group G.
Issues like this will be hard to prevent. While that doesn't mean we shouldn't work hard to make real innovations in ML, I think the legal approach of a "right to explanation" as analyzed in http://arxiv.org/pdf/1606.08813v3.pdf and recently added to European law is regardless a helpful tool to ensure accountability.
Re: Approaching fairness in machine learning
#44Earlier quoted context omitted.
I don't know what you mean by "fundamentally different" but there are definitely going to be demographic differences that the algorithm could use to predict race with good probability from hidden variables. (Where they live, for example.) History has an influence that's hard to remove from the dataset. I'd guess that another reason this problem is hard is that it's about defining the goal correctly. It's not just max…
By "fundamentally different", I mean that the most accurate model will be something like this: repayment_probability = 1 x downpayment_frac + 0.5 x credit_score + A x isBlack for some A != 0. I.e., if A = -0.2, then a black borrower with a 60% downpayment is as likely to pay back a loan as a white borrower with a 40% downpayment. If A = 0, then the bias described by tlb and danso won't occur. What you describe with h…
Repayment rates are not just individual - they also depend on the financial strength of friends and family who can help you out if you get in trouble. So, I think we have to assume that there are performance-relevant differences that an algorithm will detect.
Also, unless the dataset has information about families, this isn't based on your actual family. It's based on the average benefit people like you get from their family.
Re: Approaching fairness in machine learning
#45Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…
I keep making essentially the same point about race/gender discrimination in tech. If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X? There's no corporation that's so racist or sexist that it'll turn down saving 20% on payroll.
The issue here isn't that machine learning gives wrong answers, it's that our definition of 'fair' is irrational.
Re: Approaching fairness in machine learning
#46Earlier quoted context omitted.
Of course, this theory of discrimination is only applicable when minorities are fundamentally different from majorities . I.e., if the same ruleset is accurate for both whites and blacks (i.e., "I don't care about race, if he puts 20% down he's good"), this argument doesn't work at all - you can train your model on everyone and it'll work just fine. However, if blacks and whites need to be treated fundamentally diffe…
Lives of members of social groups can be different for historical reasons. Because of this, the best selection of features (as in, how we select and encode relevant aspects of a dataset for a particular problem) that one would need to use, as well as the correlations between them, may be different between different groups. The question is not whether there is fundamentally a difference between, in this case, racial g…
Yes, if your training data excludes relevant features then you can't use them. No one disputes this.
However, once you start including such people in your training data, these issues are not hard to prevent. In fact, ML systems will often do this accidentally even when you don't want them to (when the sign of the bias has the politically incorrect direction). It's called redundant encoding.
See the section of my blog post "What if we scrub race, but redundantly encode it?" where I do calculations to show the effect of this: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
In short, if your data is biased against a group, but you include group membership either directly or via redundant encoding, your algorithm will fix the bias as best it can.
The entire purpose of machine learning is to discover hidden features and correlations in messy data, so I fail to see why this is considered surprising.
I generally consider the "right to explanation" to be a fairly transparent attempt by the EU to keep American tech companies out of Europe. The entire purpose of ML is that it can uncover true facts that humans can't. The right to explanation is just an attempt to hobble this power, probably because few Euro companies can do it.
Re: Approaching fairness in machine learning
#47Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…
I keep making essentially the same point about race/gender discrimination in tech. If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X? There's no corporation that's so racist or sexist that it'll turn down saving 20% on payroll. The issue here isn't that machine learning gives wrong answers, it's that our definition of 'fair' is irrational.
>If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X?
Hypothetical possibility: members of group X are not perceived as 100% as effective as group Y because of pervasive bias by the employers that assumes their incompetence. They are generally perceived to be 80% as effective as a standard Y member despite actual 100% performance, and paid accordingly. A member of X needs to be 120% as effective as a Y member to be perceived at 100% Y efficiency because of stereotypes coloring their perception and an inability to objectively evaluate their performance.
Some non-hypothetical studies touching on this:
http://www.nber.org/papers/w9873.pdf http://www.pnas.org/content/109/41/16474.full.pdf+html http://advance.cornell.edu/documents/ImpactofGender.pdf http://www.socialjudgments.com/docs/Uhlmann%20and%20Cohen%20...
Re: Approaching fairness in machine learning
#48Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…
I think you're right and obviously making the machine make suboptimal decisions is definitely not a good solution. However I think a case can be made that certain protected attributes should be censored. Not to prevent the algorithm from making optimal decisions, but to prevent it from overfitting on those attributes. Which, if you think about it, is essentially what discrimination is.
If the algorithm is overfitting, it's costing its users money in the general case. Again, self righting. We don't need to hide the data we think it's overfitting on; any modern production ML system shouldn't have trouble with extraneous data. You don't see loan bots denying loans to people because they're named "Phil", for example, even though the bots have that information.
(Good) ML algorithms don't suffer from human biases; they don't know that there's a categorical difference between e.g. race and shoe size, so we don't need to hide race from these algorithms. That is, of course, unless one's explicit goal is to cripple and pessimize the algorithm for political reasons.
Re: Approaching fairness in machine learning
#49Earlier quoted context omitted.
I keep making essentially the same point about race/gender discrimination in tech. If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X? There's no corporation that's so racist or sexist that it'll turn down saving 20% on payroll. The issue here isn't that machine learning gives wrong answers, it's that our definition of 'fair' is irrational.
>If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X? Hypothetical possibility: members of group X are not perceived as 100% as effective as group Y because of pervasive bias by the employers that assumes their incompetence. They are generally perceived to be 80% as effective as a standard Y member despite actual 100% performance, and paid accordingly.…
Ideally, management would just look at the numbers at some level and figure out if there was some measurable pay disparity they could arbitrage and make money off of. I'm sure some companies have. This is a benefit of impersonal, faceless corporate structures; they don't have human qualities like biologically motivated bias in judgement. On the other hand, they don't have qualities like empathy either, so it's not clear if it's preferable or not.
Re: Approaching fairness in machine learning
#50Everyone suggesting that we ought to legislate that machines must be illogical/suboptimal is missing the point. If machine learning algorithms are unfairly discriminating against some group, then they are making sub-optimal decisions and costing their users money. This is a self-righting problem. However, a good machine learning algorithm may uncover statistical relationships that people don't like ; for example, per…
This is a great point. People believe that most groups are basically equal; this is true in the sense that if people were raised in identical environments with equal opportunities than it probably wouldn't really matter what group they were in, but wrong because that isn't the world we live in. Different groups on average experience much different environments. Machine learning doesn't care why the differences in gro…
It comes down to a choice between equality of opportunity versus equality of outcome (or some mix of the two). You can't have both - granting equal opportunities will result in unequal outcomes for all kinds of fair and unfair reasons; and ensuring equal outcomes requires unequal opportunities (e.g. quota systems).
For unfair stereotypes it's simple, you just ignore them; but there will be some group differences that are real - it would be a mighty coincidence if so many so diverse groups would magically happen to be identical in all aspects.
So it's up for the society to decide what to choose what we will do if it turns out that, other observable factors being equal, race/religion/ethnic background/etc X actually is 10% more likely to default on a loan.