Attacking discrimination with smarter machine learning • Hacker News Board

Re: Attacking discrimination with smarter machine learning

#61

Post by corey_moncure » Mon, Nov 21, 2016, 2:35 PM UTC

tomp wrote: post #49

Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

> The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this:

I'd like to add a third possible reason for your consideration. Since "criminality" i.e. guilt of committing a crime is determined after a process engaging the law enforcement and justice systems, we have to examine whether there are inherent biases in those systems that result in skewed statistics. For instance, do police officers selectively target blacks for monitoring and investigation? Are blacks discriminated against in the courtroom as a result of procedure or human nature?

Re: Attacking discrimination with smarter machine learning

#62

Post by tptacek » Mon, Nov 21, 2016, 2:36 PM UTC

alvarosm wrote: post #58

tptacek wrote: post #50

Earlier quoted context omitted.

No, I think he's more advocating for a world where we don't encode the literal status quo into computer models that make decisions for us going forward, which is something that nobody of any political stripe is likely to want.

No, no, he's advocating for a dreadful, sinister world where a bunch of high priests strip you of your freedom to decide how you conduct business... How much "discrimination" will be good enough in the end? I'll tell you what the endgame is: no discrimination at all, everybody gets a loan. So, in other words, no loans at all because nobody would conduct business in those conditions.

First, I'm pretty sure he's not.

Second, there's no political orientation I can think of that is entirely comfortable with the status quo. Conservatives believe themselves to be discriminated against, just in a different set of circumstances. The underlying problem with ML is conceptual, not political.

Re: Attacking discrimination with smarter machine learning

#63

Post by GFK_of_xmaspast » Mon, Nov 21, 2016, 2:43 PM UTC

riskable wrote: post #40

tptacek wrote: post #20

Earlier quoted context omitted.

That is a story that banks like to believe about themselves, but it's worth considering that emergent discrimination (and the broader bucket of emergent malfeasance) is a property of all complex systems, not just machine learning. So, for instance, Wells Fargo needed to do more than just hope that its trustworthiness would survive its incentive systems.

Not taking steps to prevent emergent discrimination or malfeasance is careless. Do you think the board of directors wants to find out that the CEO intentionally ignored calls to the ethics hotline? That he didn't take steps to ensure whistleblowers weren't punished? The consequences of these things are very bad for stock prices! This is doubly true for banks which take reputational risk far more seriously than most b…

Wells Fargo stock appears to be back up to where it was a few months ago

Re: Attacking discrimination with smarter machine learning

#64

Post by losvedir » Mon, Nov 21, 2016, 2:44 PM UTC

tptacek wrote: post #24

mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

I don't get the impression that most people are afraid of irrelevant correlations, but rather relevant ones that nevertheless disproportionately affect certain populations.

To use your Armenian example, it could be that while being Armenian doesn't actually affect your driving, a "true" model could still end up being bad for Armenians if being Armenian is correlated with the things that actually do affect crash risk.

But what about this: we don't need to solve all social ills every single time. What if we let the algorithm correctly decide crash risk, and if we notice that it unduly impacts Armenians and that's not an outcome we want, we via a separate channel compensate the Armenians? That is, acknowledge the fact that Armenians may crash more, but give them a government subsidy to offset the higher premiums, and work to bring the premiums down (i.e.: fix the underlying issues that being Armenian is correlated with).

It's related to something I've been thinking about lately with regard to minimum wage. I like the idea that everyone should have a livable income, but tying the implementation to businesses that have low wage jobs seems like mixing concerns. For example, my company doesn't have any minimum wage jobs, but shouldn't my company chip into this social ideal same as any other?

What if we let the businesses pay whatever the market will bear, and if we decide that as a social concern people should get more than that, we subsidize them from the government, which is wear this concern is coming from in the first place.

Re: Attacking discrimination with smarter machine learning

#65

Post by icebraining » Mon, Nov 21, 2016, 2:45 PM UTC

Dowwie wrote: post #2

For more information about "ethics and algorithms", read: "Weapons of Math Destruction" [1] or at least listen to the EconTalk podcast between the author and host Russ Roberts [2] [1] https://www.amazon.com/Weapons-Math-Destruction-Increases-In... [2] http://www.econtalk.org/archives/2016/10/cathy_oneil_on_1.ht...

Previous discussion on the podcast episode: https://news.ycombinator.com/item?id=12642432

Re: Attacking discrimination with smarter machine learning

#66

Post by weberc2 » Mon, Nov 21, 2016, 2:47 PM UTC

I agree that this should be a possibility, while I think the odds that millions of law enforcement officers and criminal justice faculty would spontaneously conspire against a particular race, I also think it's more likely than some genetic predisposition toward crime (i.e., the first option).

Re: Attacking discrimination with smarter machine learning

#67

Post by zmoreira » Mon, Nov 21, 2016, 2:48 PM UTC

corey_moncure wrote: post #61

tomp wrote: post #49

Earlier quoted context omitted.

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

> The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: I'd like to add a third possible reason for your consideration. Since "criminality" i.e. guilt of committing a crime is determined after a process engaging the law enforcement and justice systems, we have to examine whether there are inherent biases in those systems that result in ske…

This is studiable and has been studied. A study regarding police killings, for example:

Do White Police Officers Unfairly Target Black Suspects?

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2870189&...

Using a unique data set we link the race of police officers who kill suspects with the race of those who are killed across the United States. We have data on a total of 2,699 fatal police killings for the years 2013 to 2015. This is 1,333 more killings by police than is provided by the FBI data on justifiable police homicides. When either the violent crime rate or the demographics of a city are accounted for, we find that white police officers are not significantly more likely to kill a black suspect. For the estimates where we know the race of the officer who killed the suspect, the ratio of the rate that blacks are killed by black versus white officers is large — ranging from 3 to 5 times larger. However, because the media may under report the officer’s race when black offic-ers are involved, other results that account for the fact that a disproportionate number of the un-known race officers may be more reliable. They indicate no statistically significant difference be-tween killings of black suspects by black and white officers. Our panel data analysis that looks at killings at the police department level confirms this. These findings are inconsistent with taste-based racial discrimination against blacks by white police officers. Our estimates examining the killings of white and Hispanic suspects found no differences with respect to the races of police officers. If the police are engaged in discrimination, such discriminatory behavior should also be more difficult when body or other cameras are recording their actions. We find no evidence that body cameras affect either the number of police killings or the racial composition of those killings.

Re: Attacking discrimination with smarter machine learning

#68

Post by zepto » Mon, Nov 21, 2016, 2:52 PM UTC

weberc2 wrote: post #66

I agree that this should be a possibility, while I think the odds that millions of law enforcement officers and criminal justice faculty would spontaneously conspire against a particular race, I also think it's more likely than some genetic predisposition toward crime (i.e., the first option).

Nobody is talking about a conspiracy - racism is about bias.

Re: Attacking discrimination with smarter machine learning

#69

Post by cwilkes » Mon, Nov 21, 2016, 2:53 PM UTC

tptacek wrote: post #24

mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

In your example there could be deeper reasons for Armenians having higher crash rates. Let's say it is a combination of zip code, the food they are eating from a local Armenian Bell, and the phase of the moon.

However with a lot of ML classes we are told to look for the simpler explanation which would be nationality in this case, in the absence of collection of the real reasons.

Not sure how that can be gotten around. I'll read the paper.

Re: Attacking discrimination with smarter machine learning

#70

Post by [deleted] » Mon, Nov 21, 2016, 2:54 PM UTC

tomp wrote: post #49

Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups. For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly…

[deleted]