Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

91–100 of 201 posts

Re: Attacking discrimination with smarter machine learning

#91

Post by d--b »

icebraining wrote: post #88
d--b wrote: post #8

Nice to see that the debate has reached the ears of the main people working on this field. What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve. In other words, the objective function of the optimization problem needs not only match the current state of the world, and provide an hindsight in one's own economic interests, it also needs to take into account th…

I'm not sure that's clear. There are actually two ways to achieve the outcome we want; tweaking the model or changing the inputs . What I mean, say the model identifies that a certain group has a greater risk due to systemic problems. If you change something about the group, you can change the calculated risk without changing the model. And this may very well be a better way to achieve the outcome you want. Specifica…

No I don't think I expressed myself clearly. The problem is that machine learning learns from biased data. The input data is the total US population, which has wealthier groups than others. Because of the way we train machine learning (using all the data we have), the bias that is in the original data gets transfered to the logic.

Specifically, a machine learning will produce different numbers for 2 individuals with the exact same characteristics except the race. And that is the problem that needs to be addressed.

Let's put it in another context. Let's say I'm a white athlete, and I'm very good at running the 100m race. Actually I run just as fast as a black person who is my main rival. Now if someone has to select one of us to go to the Olympics, they should toss a coin to decide who goes. If you use a ML algorithm, it would absolutely send the black person, because no white people has won a 100m race in the last 20 olympics. That's the kind of bias ML does and that needs to be addressed.

Re: Attacking discrimination with smarter machine learning

#92

Post by KirinDave »

tjic wrote: post #29
tptacek wrote: post #24

Earlier quoted context omitted.

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Please don't say "we". Not everyone shares your politics. I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

I've run into many people who share this sentiment, and it always surprises me. I've never once met a person who was cheerful about experiencing algorithm-driven prevenge. I think it's very easy to sit back and say "I think we should let this happen!" if they've never knowingly experienced loss based on this phenomenon.

A great example is how very resentful many young white men of college age are that universities are requiring them to take sensitivity courses designed to reduce the instance of campus rape, but strictly speaking men of that age are the overwhelming majority of bad actors in that environment. Statistically and logistically speaking, it's smarter and cheaper to just require all men of college age to take courses reminding them that rape is not okay rather than dealing with the moral, legal and healthcare costs of the alternative.

In some cases, our relatively primitive algorithms pick up on correlations that should not be acted on because we're actively working to correct them. For example, it would be inappropriate to pre-reject job applications based on skin color if in a certain culture, it's less likely for that person to have a college degree.

Even if that insight is correct, it's usually part of something that society hopes to correct or that applicants should be given the benefit of the doubt about, otherwise very serious negative responses will emerge.

Acting on existing categories may reinforce. It may not. For now, it's a case-by-case basis we'll have to act on. Maybe one day, modeling techniques and data sources will become sophisticated and robust enough to make every decision for us. That day is not today.

Re: Attacking discrimination with smarter machine learning

#93

Post by [deleted] »

tjic wrote: post #29
tptacek wrote: post #24

Earlier quoted context omitted.

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Please don't say "we". Not everyone shares your politics. I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

[deleted]

Re: Attacking discrimination with smarter machine learning

#94

Post by [deleted] »

tjic wrote: post #29
tptacek wrote: post #24

Earlier quoted context omitted.

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Please don't say "we". Not everyone shares your politics. I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

[deleted]

Re: Attacking discrimination with smarter machine learning

#95

Post by [deleted] »

tjic wrote: post #29
tptacek wrote: post #24

Earlier quoted context omitted.

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Please don't say "we". Not everyone shares your politics. I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

[deleted]

Re: Attacking discrimination with smarter machine learning

#96

Post by lifeisstillgood »

sorry, just checking - young black men are statisically less likely to rape than equivalent white (young men)

That is surprising. And given this discussion about only wanting stats that have a valid explanation as well as fitting the facts, is interesting.

Re: Attacking discrimination with smarter machine learning

#97

Post by cs2818 »

What strikes me as interesting in this example is that it seems to assume the classifier is run once and that its decisions have no effect on future decisions.

I would imagine that if we are separating people into groups based on demographic or social factors to make decisions, then those decisions may have an impact on that entire group. (In this example, maybe granting more loans to the blue group alters the group characteristics and leads to higher profit in the long term despite higher immediate risks).

Is there an area of ML research that considers this kind of concept?

Re: Attacking discrimination with smarter machine learning

#98

Post by witty_username »

mtgx wrote: post #71
angry-hacker wrote: post #48

Earlier quoted context omitted.

Which made the insurance for women, who are traditionally safer drivers because not taking unless risks, go up. Well facts can't be racist but they are at the same time.

But if you were a man who was an excellent driver, would you like it that you get to pay more just because you were born into a gender that was "more reckless"? Plus, women want equal rights, so why not be equal in this, too? I think for things like insurance, or healthcare taxes (the model adopted by most countries) it's fair to normalize the price for everyone. Otherwise, the system doesn't really work.

Price of insurance is not a right.

It is not fair to normalize the price. It penalizes people who are healthy and good drivers (for example not smoking and speeding).

Re: Attacking discrimination with smarter machine learning

#99

Post by zitterbewegung »

cs2818 wrote: post #97

What strikes me as interesting in this example is that it seems to assume the classifier is run once and that its decisions have no effect on future decisions. I would imagine that if we are separating people into groups based on demographic or social factors to make decisions, then those decisions may have an impact on that entire group. (In this example, maybe granting more loans to the blue group alters the group…

I don't think there is though it may fall under ethical considerations of AI or economics of AI. Would be hard to do a study since AI has really only been around for less than 50 years.

Re: Attacking discrimination with smarter machine learning

#100

Post by [deleted] »

KirinDave wrote: post #92
tjic wrote: post #29

Earlier quoted context omitted.

Please don't say "we". Not everyone shares your politics. I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

I've run into many people who share this sentiment, and it always surprises me. I've never once met a person who was cheerful about experiencing algorithm-driven prevenge. I think it's very easy to sit back and say "I think we should let this happen!" if they've never knowingly experienced loss based on this phenomenon. A great example is how very resentful many young white men of college age are that universities ar…

[deleted]
Post reply on HN