Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

151–160 of 201 posts

Re: Attacking discrimination with smarter machine learning

#151

Post by yummyfajitas »

tptacek wrote: post #122
BickNowstrom wrote: post #82

Earlier quoted context omitted.

It is possible to inspect black box models. See for instance http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf and https://homes.cs.washington.edu/~marcotcr/blog/lime/ . It is also common to use highly accurate white box models for cases where it is important to not overfit, leak, or discriminate, like MARS and GA^2M (somewhat out of scope for an actuary using R). If a computer program spots an irrelevant unhelp…

Isn't that the kind of work we're commenting on with this story?

No, it's not at all the same. Those articles are about eliminating overfitting and improving accuracy. This article is about how much fairness constraints will reduce your accuracy/objective function.

Re: Attacking discrimination with smarter machine learning

#152

Post by [deleted] »

tptacek wrote: post #122
BickNowstrom wrote: post #82

Earlier quoted context omitted.

It is possible to inspect black box models. See for instance http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf and https://homes.cs.washington.edu/~marcotcr/blog/lime/ . It is also common to use highly accurate white box models for cases where it is important to not overfit, leak, or discriminate, like MARS and GA^2M (somewhat out of scope for an actuary using R). If a computer program spots an irrelevant unhelp…

Isn't that the kind of work we're commenting on with this story?

[deleted]

Re: Attacking discrimination with smarter machine learning

#153

Post by qb45 »

yummyfajitas wrote: post #142
tptacek wrote: post #24

Earlier quoted context omitted.

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

No, bogus correlations and inaccurate conclusions are explicitly NOT the computer science problem being worked on here. You are trying to frame the problem as algorithms wrongly classifying people. That's just a standard statistics problem (called "overfitting") and has nothing to do with fairness. The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much…

I think what the research itself (as opposed to politics surrounding it) was about is summed up in the last paragraph:

A key result in the paper by Hardt, Price, and Srebro shows that—given essentially any scoring system—it's possible to efficiently find thresholds that meet any of these criteria.

The crucial thing is that this "credit score" doesn't actually predict defaults quite well, because blues who are 50% likely to default have higher score than oranges who are 50% likely to default. If you base decision on a simple credit score cutoff, the blue defaulters will screw you up because of their higher score. You can defend from this by tweaking cutoffs per-group to achieve profit maximization or various "fairness" goals. In a sense, it's an attempt at producing non-garbage out from garbage in.

And of course whether real-world implementations of this idea are "good" or "bad" is another can of worms altogether and varies case by case. They probably could have came up with some better examples than credit scores and labeling people by colors.

Re: Attacking discrimination with smarter machine learning

#154

Post by yummyfajitas »

qb45 wrote: post #153
yummyfajitas wrote: post #142

Earlier quoted context omitted.

No, bogus correlations and inaccurate conclusions are explicitly NOT the computer science problem being worked on here. You are trying to frame the problem as algorithms wrongly classifying people. That's just a standard statistics problem (called "overfitting") and has nothing to do with fairness. The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much…

I think what the research itself (as opposed to politics surrounding it) was about is summed up in the last paragraph: A key result in the paper by Hardt, Price, and Srebro shows that—given essentially any scoring system—it's possible to efficiently find thresholds that meet any of these criteria. The crucial thing is that this "credit score" doesn't actually predict defaults quite well, because blues who are 50% lik…

Let me repeat what I said: Go play with the simulation to see. The various fairness criteria all achieve lower than maximal profits.

The best predictor is one which is explicitly discriminatory based on race: it takes both FICO score and race into account. There are worse predictors which also discriminate explicitly based on race, but in some "fair" way.

Finally, the worst predictor throws away directly relevant racial information.

This is not about garbage input data - that's not the problem being addressed here at all. The input data is perfectly fine. The problem is just that the input data says "race discrimination is the best way to make money".

Re: Attacking discrimination with smarter machine learning

#155

Post by weberc2 »

Retric wrote: post #106
weberc2 wrote: post #84

Earlier quoted context omitted.

I wouldn't make assumptions, but I don't understand your point.

Sounds like you are just avoiding uncomfortable questions. However, most people will make assumption on race and class from things like PCP even though zero information about that in my example.

No, I just don't understand your question. Do you want me to make a crappy guess based on the crappy data you provided? If so, I don't understand why... Judging by subthreads, you just want to say "Ha! Your crappy guess was wrong!". If you want me to pretend that I'm somehow involved in the processing of this case, but I'm restricted to only the crappy data provided, I wouldn't "assume" anything--I would say I don't have enough information to establish any individual's guilt much less sentence them.

Re: Attacking discrimination with smarter machine learning

#156

Post by weberc2 »

zepto wrote: post #68
weberc2 wrote: post #66

I agree that this should be a possibility, while I think the odds that millions of law enforcement officers and criminal justice faculty would spontaneously conspire against a particular race, I also think it's more likely than some genetic predisposition toward crime (i.e., the first option).

Nobody is talking about a conspiracy - racism is about bias.

Fair enough. It would be unlikely that millions of police would be spontaneously biased against African Americans, particularly when you factor in that those officers most likely to use excessive force against African Americans are themselves African American.

Re: Attacking discrimination with smarter machine learning

#157

Post by tjic »

tptacek wrote: post #127
tjic wrote: post #125

Earlier quoted context omitted.

chronic6l has a wealth of peer-reviewed science on his side, tptacek. While I imagine that most of us think it would be nicer if all subgroups had the same mean attributes, that's not the world we live in and we shouldn't turn our backs on science and embrace faith-based arguments, no matter how nice the motivations.

No, he doesn't. He has a mixture of early Jensenist psychometric research that has been superseded, and neo-phrenologists like Rushton that have been discredited. But of course it's easy to drop little bombs like this into threads and put the onus on other people to explain the science. That's what makes it trolling. Whatever snappy response you have for this, please spare us. There is a reason "the subject of racial…

> But of course it's easy to... put the onus on other people to explain the science.

> Whatever snappy response you have for this, please spare us.

In two sentences you simultaneously attacked me for not providing cites and then told me not to provide cites.

If you're going to Kafka trap people like this, where there is literally no way to respond to you without being attacked, that doesn't look like good faith debate, and I'm going to disengage.

Re: Attacking discrimination with smarter machine learning

#158

Post by weberc2 »

sangnoir wrote: post #86
zmoreira wrote: post #67

Earlier quoted context omitted.

This is studiable and has been studied. A study regarding police killings, for example: Do White Police Officers Unfairly Target Black Suspects? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2870189&... Using a unique data set we link the race of police officers who kill suspects with the race of those who are killed across the United States. We have data on a total of 2,699 fatal police killings for the years…

> This is studiable and has been studied True, but there has been more than one paper written on the subject, which don't all agree with the one you linked vis over-representation for crimes. The black/white marijuana arrest gap, in nine charts https://www.washingtonpost.com/news/wonk/wp/2013/06/04/the-b... As you're probably aware, black Americans are arrested for marijuana possession far more frequently than whites…

Do the findings control for which group uses more in public? My experience living in Chicago and Waterloo, Iowa (the "blackest" city in Iowa--http://www.iowadatacenter.org/Publications/aaprofile2016.pdf) suggests that this disparity alone could explain the disparity in arrest rates. It's quite possible that there's a cultural difference in drug abuse that could skew the results.

Re: Attacking discrimination with smarter machine learning

#160

Post by mcbits »

witty_username wrote: post #98
mtgx wrote: post #71

Earlier quoted context omitted.

But if you were a man who was an excellent driver, would you like it that you get to pay more just because you were born into a gender that was "more reckless"? Plus, women want equal rights, so why not be equal in this, too? I think for things like insurance, or healthcare taxes (the model adopted by most countries) it's fair to normalize the price for everyone. Otherwise, the system doesn't really work.

Price of insurance is not a right. It is not fair to normalize the price. It penalizes people who are healthy and good drivers (for example not smoking and speeding).

The whole purpose of insurance is to normalize the price. Without insurance, people who take the most risks with their health/driving would be more likely to pay (or at least owe) millions, while the health-conscious and risk-averse would generally pay much closer to $0 (until they get old or unlucky).

It's interesting that insurance premiums can act as behavioral incentives in some cases, but I doubt it has ever led someone to change their gender or race.

Post reply on HN