Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

41–50 of 201 posts

Re: Attacking discrimination with smarter machine learning

#41

Post by reader5000 »

theBobBob wrote: post #23
mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

Interestingly the EU has banned insurance underwriting based on gender even with all the actuarial data backing it up. Age is still fair game though.

So (assuming males are more costly to insure) either (a) females pay more than they should and are effectively subsidizing males or (b) males pay less than they should and the insurance companies will go broke..

Re: Attacking discrimination with smarter machine learning

#43

Post by [deleted] »

tjic wrote: post #32
tptacek wrote: post #17

Earlier quoted context omitted.

It's referring to the much-talked-about effect of emergent discrimination, where the model fitting process has the effect of amplifying the status quo, despite the fact that the status quo is informed in large part by structural injustices. For (oversimplified) instance: poor black people represent a cohort of loan applicants likely to default, and the model fitting process may go a step worse and attribute "default…

> despite the fact that the status quo is informed in large part by structural injustices. This is presented as if it's an unambiguous fact, when it's largely a political stance.

[deleted]

Re: Attacking discrimination with smarter machine learning

#44

Post by raverbashing »

Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

Well, "theoretically" you're right

In practice, the first variables to be used for classification are the ones that have a biggest effect. Then you're going to use them (from order of importance) as eliminatory or classificatory

> because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me.

But the opposite is also discriminatory, which is what the article is showing. Because then you're using the same ruler to evaluate different groups, and of course the minority person with 2 jobs can't match the credit score of an Ivy-League educated WASP

Re: Attacking discrimination with smarter machine learning

#45

Post by dragonwriter »

tjic wrote: post #32
tptacek wrote: post #17

Earlier quoted context omitted.

It's referring to the much-talked-about effect of emergent discrimination, where the model fitting process has the effect of amplifying the status quo, despite the fact that the status quo is informed in large part by structural injustices. For (oversimplified) instance: poor black people represent a cohort of loan applicants likely to default, and the model fitting process may go a step worse and attribute "default…

> despite the fact that the status quo is informed in large part by structural injustices. This is presented as if it's an unambiguous fact, when it's largely a political stance.

It's an unambiguous fact that the status quo is shaped heavily be many generations of de jure discrimination including chattel slavery and continting structural inequalities in political power that still exist that were designed to protect those other unequal institutions.

It's a subjective political view that any or all of those things are injustices, of course, since justice is a subjective thing.

Re: Attacking discrimination with smarter machine learning

#47

Post by tptacek »

dragonwriter wrote: post #45
tjic wrote: post #32

Earlier quoted context omitted.

> despite the fact that the status quo is informed in large part by structural injustices. This is presented as if it's an unambiguous fact, when it's largely a political stance.

It's an unambiguous fact that the status quo is shaped heavily be many generations of de jure discrimination including chattel slavery and continting structural inequalities in political power that still exist that were designed to protect those other unequal institutions. It's a subjective political view that any or all of those things are injustices, of course, since justice is a subjective thing.

This happens to be something I believe strongly, but the comment you're responding to doesn't make sense even if you strongly disagree with that, as it supposes we can snapshot American society as it is in 2016 and synthesize from it reliably just and sensible decisions. Nobody believes this, no matter what their politics (unless there's a "status-quo-ism" I'm unaware of).

The problem (or at least, one of the more important problems) being addressed in this work is the unintended amplification of the status quo --- the implicit notion that if something is a certain way now, it is best that it always be that way.

Re: Attacking discrimination with smarter machine learning

#48

Post by angry-hacker »

theBobBob wrote: post #23
mvindahl wrote: post #22

Well. At the end of the day, the companies will pick thresholds and rates to maximize their profits, based upon the data that they have available. They don't know every detail of our personal lives -- which would also be kind of unsettling -- so they have to resort to a simplified picture. Simplifications are always prejudiced at the individual level but if the prejudice is reflected in the numbers, it's balanced. It…

Interestingly the EU has banned insurance underwriting based on gender even with all the actuarial data backing it up. Age is still fair game though.

Which made the insurance for women, who are traditionally safer drivers because not taking unless risks, go up. Well facts can't be racist but they are at the same time.

Re: Attacking discrimination with smarter machine learning

#49

Post by tomp »

Scea91 wrote: post #25

>[...] concept called equal opportunity. Here, the constraint is that of the people who can pay back a loan, the same fraction in each group should actually be granted a loan. This does not seem fair to me, because if this is applied then your race (group) would determine your credit score threshold which feels discriminatory to me. I feel that, by definition, it is not discriminatory only if none of your attributes…

I think it's useful to figure out why there are discrepancies between two groups.

For example, let's take blacks in the US. The data tells you that a black person is more likely to be a criminal than a white person. There are two possible reasons for this: (1) blacks are more prone to crime, or (2) blacks are more likely to live in circumstances that make them criminals. With access to only anecdotal data, I strongly believe that (2) is true, and that if you took into account enough circumstances (e.g. single parent, school district, income level, parents' wealth) you'd be able to remove race from your model and still arrive to the "equal opportunity" result. That way, you wouldn't discriminate based on race, but you would still help the people most needy of help (poor, uneducated, etc.). I think the same applies to colege applications and sex wage gap, which is why I strongly oppose any kind of affirmative action.

Life expentancy sex gap might be a different case. IIRC, men die earlier because of some behavioural/social tendencies (working dangerous jobs, supressing emotions, risky behaviour (speeding, smoking)), so maybe there are some non-discriminatory (or at least less-discriminatory) ways of determining life expectancy (e.g. testosterone level, job description, ...). However, there are also biological differences that seem very strongly linked to the quality of being male (i.e. the Y chromosome). E.g. prostate cancer is less lethal than breast cancer, and women are more likely to get MS than men. These particular examples suggest that men should live longer, but I'm guessing there might be other gender-specific ilnesses that might reduce their (our) expected lifespan. If that's the case, I don't think it's too discriminatory to have different insurance fees for different sexes.

In all such scenarios, however, there could still be broader societal goals that would override specific instances of (non-)discrimination. For exapmle, AFAIK women's health is more expensive (because of pregnancy), but having more children benefits everyone in the society (in the West), so it makes sense to "discriminate" against men by letting women pay less for their health insurance.

Re: Attacking discrimination with smarter machine learning

#50

Post by tptacek »

alvarosm wrote: post #46

You realize you're advocating totalitarianism, right?

No, I think he's more advocating for a world where we don't encode the literal status quo into computer models that make decisions for us going forward, which is something that nobody of any political stripe is likely to want.
Post reply on HN