Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

21–30 of 201 posts

Re: Attacking discrimination with smarter machine learning

#21

Post by [deleted] »

denzil_correa wrote: post #16
d--b wrote: post #8

Nice to see that the debate has reached the ears of the main people working on this field. What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve. In other words, the objective function of the optimization problem needs not only match the current state of the world, and provide an hindsight in one's own economic interests, it also needs to take into account th…

> What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve Good point. Systems should work for people, not the other way round.

[deleted]

Re: Attacking discrimination with smarter machine learning

#22

Post by mvindahl »

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It may not feel perfectly just but at least it's more just than an imbalanced prejudice.

A real life example: In many (all?) countries, it's common to pay more for your car insurance if you are young, if you are male, or if you recently acquired your driver's license. One might frame this as a prejudice towards young, male drivers as being reckless. But statistically they are just that, so the prejudice is balanced (within a reasonable tolerance). It's unfair to the careful young, male driver, but well, life is not always fair.

Re: Attacking discrimination with smarter machine learning

#23

Post by theBobBob »

mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

Interestingly the EU has banned insurance underwriting based on gender even with all the actuarial data backing it up. Age is still fair game though.

Re: Attacking discrimination with smarter machine learning

#24

Post by tptacek »

mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a computer program might spot some irrelevant correlation that determines that Armenians shouldn't get car loans --- and we're especially not comfortable with the idea that we might learn that this has happened only after years of unintended discrimination, because there is no straightforward way to interrogate the model for every possible bogus correlation it may have snagged on.

That's the computer science problem being worked on here.

Re: Attacking discrimination with smarter machine learning

#25

Post by Scea91 »

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan.

This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me.

I feel that, by definition, it is not discriminatory only if none of your attributes that you could be discriminated against (gender, race, ...) are taken into account at all.

Maybe the concept of 'equal opportunity' is just some compromise between discrimination and making less informed decisions.

Re: Attacking discrimination with smarter machine learning

#26

Post by pdkl95 »

tptacek wrote: post #7
lugus35 wrote: post #3

"Discrimination" is the marxist way of saying "Choice".

This empty snark would make more sense if you weren't commenting on machine learning, where the problem is emergent discrimination unintended by the designers --- in other words, discrimination that isn't chosen . As it stands, all you've done here is reveal to the thread that you don't understand why discrimination is a concern with machine learning, but you have very strong feelings about it anyways. Congratulation…

> unintended

While unintended discrimination emerging from machine learning techniques is a very important problem that needs to be addresses, it's also important to remember that practices such as "redlining" show that there is still an unfortunate amount of intentional discrimination. The complexity of machine learning creates a lot of opportunity to hide intentionally biased features.

Re: Attacking discrimination with smarter machine learning

#27

Post by tptacek »

pdkl95 wrote: post #26
tptacek wrote: post #7

Earlier quoted context omitted.

This empty snark would make more sense if you weren't commenting on machine learning, where the problem is emergent discrimination unintended by the designers --- in other words, discrimination that isn't chosen . As it stands, all you've done here is reveal to the thread that you don't understand why discrimination is a concern with machine learning, but you have very strong feelings about it anyways. Congratulation…

> unintended While unintended discrimination emerging from machine learning techniques is a very important problem that needs to be addresses, it's also important to remember that practices such as "redlining" show that there is still an unfortunate amount of intentional discrimination. The complexity of machine learning creates a lot of opportunity to hide intentionally biased features.

A related term here is "mathwashing".

Re: Attacking discrimination with smarter machine learning

#28

Post by alvarosm »

This is how it should be: "Max Profit. The most profitable, since there are no constraints. But the two groups have different thresholds, meaning they are held to different standards."

Here's the big fallacy: "the two groups have different thresholds, meaning they are held to different standards." They are not held to different standards because they're different groups, but because of other reasons that indicate different loan default rates. So you cannot call this "discrimination". This is how things should be.

Re: Attacking discrimination with smarter machine learning

#29

Post by tjic »

tptacek wrote: post #24
mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Please don't say "we".

Not everyone shares your politics.

I'm perfectly happy with algorithms detecting that certain people are more likely to be safe drivers than average, and giving them lower rates, and concentrating premiums on the groups more likely to be in accidents, even if I don't understand why Armenians (in your example) get in more crashes.

Re: Attacking discrimination with smarter machine learning

#30

Post by Scea91 »

tptacek wrote: post #24
mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

The difference with machine learning is that the model isn't designed by humans, through an actuarial process we can keep our brains wrapped around. It's a black box. We are OK attributing "crash risk" to young male drivers, because we can observe both that they are as a cohort statistically likely to crash and also understand why that would be the case. On the other hand, we're not comfortable with the idea that a c…

Depends on the model. Decision trees for example are white box models. In scenarios like this white box models are always preferred.
Post reply on HN