Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attacking discrimination with smarter machine learning

research.google.com

Post reply on HN
Print view View original

11–20 of 201 posts

Re: Attacking discrimination with smarter machine learning

#11

Post by riskable »

amelius wrote: post #4

But how can we find out if a company uses ML in a non-discriminating way? If we cannot see it, and check it, then there is no incentive for companies to use it. My guess is that at most companies spending time on making an algorithm non-discriminating will be viewed as a waste of time and money.

I work for a big bank... As long as people freak out when banks are found to be discriminating they will do their best to not discriminate.

Banks are built on trustworthiness. Having your bank's name in the headlines for discriminatory practices can have a severe negative impact on trustworthiness. They have a whole teams of people devoted to this topic and every year at most banks every employee has to learn about, "reputational risk."

Having worked at big banks for over a decade now I am 90% certain that discrimination by banks at this point is primarily due to carelessness.

Re: Attacking discrimination with smarter machine learning

#13

Post by dmichulke »

lugus35 wrote: post #3

"Discrimination" is the marxist way of saying "Choice".

I think he means something that I quoted in an AirBnb thread a 73 days ago (shameless self-quoting)

Whenever you

- book a flight, - buy bread or milk, or - get a girl friend

you discrimate in favor of one and against all others. Subjective discrimination is the only way to choose, discrimination is the sure consequence of your every choice.

I don't get the "marxist" part though

update: Of course, forgive my bout of idiocy

Re: Attacking discrimination with smarter machine learning

#14

Post by sergioisidoro »

I've played around with this concept, trying to replicate some previous work [1].

It's a sensitive topic, because sometimes we're actually tampering with the data, trying to eliminate known human or selection biases.

The first defence against discrimination is, in my honest oppinion, for everyone working with data to be aware of these problems. To know that, besides ROC, precisions and recalls, we should measure the impacts of the models in sensitive demographics (gender, race, nationality, sexuality).

And one of the things that I learned (in [1]) is that, even if you're carfull with the features you use, you might still have a negative effect.

[1] https://github.com/sergioisidoro/aequalis

Re: Attacking discrimination with smarter machine learning

#15

Post by denzil_correa »

> Restricting to equal opportunity thresholds transfers the "burden of uncertainty" away from these groups and onto the creators of the scoring system. Doing so provides an incentive to invest in better classifiers.

Personally, I find this as the key outcome here. The accountability is on the people/systems who make the decision and that leads to an appropriate incentive. Win-win as a start.

Re: Attacking discrimination with smarter machine learning

#16

Post by denzil_correa »

d--b wrote: post #8

Nice to see that the debate has reached the ears of the main people working on this field. What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve. In other words, the objective function of the optimization problem needs not only match the current state of the world, and provide an hindsight in one's own economic interests, it also needs to take into account th…

> What is important to note here is that we need to tweak the mathematical model to the culture we want to achieve

Good point. Systems should work for people, not the other way round.

Re: Attacking discrimination with smarter machine learning

#17

Post by tptacek »

beejiu wrote: post #12

Presumably it means unlawful or unethical discrimination, since classification without discrimination doesn't make sense.

It's referring to the much-talked-about effect of emergent discrimination, where the model fitting process has the effect of amplifying the status quo, despite the fact that the status quo is informed in large part by structural injustices. For (oversimplified) instance: poor black people represent a cohort of loan applicants likely to default, and the model fitting process may go a step worse and attribute "default risk" to all black people.

The key thing to understand is that we're talking about discrimination that is usually unintended and unexpected by the designers of these systems.

Re: Attacking discrimination with smarter machine learning

#18

Post by denzil_correa »

sergioisidoro wrote: post #14

I've played around with this concept, trying to replicate some previous work [1]. It's a sensitive topic, because sometimes we're actually tampering with the data, trying to eliminate known human or selection biases. The first defence against discrimination is, in my honest oppinion, for everyone working with data to be aware of these problems. To know that, besides ROC, precisions and recalls, we should measure the…

> And one of the things that I learned (in [1]) is that, even if you're carfull with the features you use, you might still have a negative effect.

One needs to understand how these features interplay with each other. For example, you may not directly use a protected class feature (race) to make your prediction but you might end up using a secondary or tertiary variable (like location) to end up learning a protected class feature due to statistical correlations.

Re: Attacking discrimination with smarter machine learning

#20

Post by tptacek »

riskable wrote: post #11
amelius wrote: post #4

But how can we find out if a company uses ML in a non-discriminating way? If we cannot see it, and check it, then there is no incentive for companies to use it. My guess is that at most companies spending time on making an algorithm non-discriminating will be viewed as a waste of time and money.

I work for a big bank... As long as people freak out when banks are found to be discriminating they will do their best to not discriminate. Banks are built on trustworthiness. Having your bank's name in the headlines for discriminatory practices can have a severe negative impact on trustworthiness. They have a whole teams of people devoted to this topic and every year at most banks every employee has to learn about,…

That is a story that banks like to believe about themselves, but it's worth considering that emergent discrimination (and the broader bucket of emergent malfeasance) is a property of all complex systems, not just machine learning. So, for instance, Wells Fargo needed to do more than just hope that its trustworthiness would survive its incentive systems.
Post reply on HN