EU regulations on algorithmic decision-making and a “right to explanation” • Hacker News Board

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#111

Post by DominikR » Thu, Jul 07, 2016, 11:19 PM UTC

InTheArena wrote: post #110

DominikR wrote: post #92

Earlier quoted context omitted.

> It's my PoV that It's not predictability versus red-tape, and the people trying to do unaccountable analytics in this space are (perhaps in-advertedily) perpetuating racism. I was with you until that last line. It is not racism to provide better credit conditions to groups that demonstrably have a lower risk of defaulting on a credit. It would be racism if you offer worse conditions to a certain group without any r…

You said: I was with you until that last line. It is not racism to provide better credit conditions to groups that demonstrably have a lower risk of defaulting on a credit. It absolutely is, under ECOA, which explicitly maintains that you may not discriminate by race for loans. And btw, as someone very familiar with credit scoring once mentioned to me, a small part of the reason the credit scores do not take into acc…

I didn't say that it is okay to discriminate by race for anything, I'm against that.

What I'm saying is that people should be treated fairly instead of using some kind of affirmative action.

If you did a proper verification of income, assets, liabilities and education, you'd likely find that some groups would have to receive worse credit conditions than others.

This doesn't mean that the system is or would be racist, it would strictly evaluate on socio-economic background, not based on race.

What I reject is to handicap one group to make up for the losses that another group generates for a bank by artificially lowering standards for certain groups.

Although this is made with best intentions, it is inherently racist (it presumes that some races are inferior to others - hence the introduction of lower standards), whereas evaluating the way I described would not be racist.

Btw: I'm myself a second-generation minority. If I'd have had the option to navigate through life with some social security autopilot and affirmative action provided by society I would have never made it to the upper strata of society.

You have to understand that expectations for life from where I and many others started were much lower than for most others, so people from this level are much more willing to accept a lower living standard provided for free than working hard for the chance to a higher standard.

From the perspective of upper middle class people these kinds of affirmative action and social benefits might seem like genuinely helping people whereas from my perspective it seems like poison that could have had inhibited my will to work on myself and fight to move upwards in society.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#112

Post by Houshalter » Fri, Jul 08, 2016, 1:01 AM UTC

yummyfajitas wrote: post #99

Houshalter wrote: post #97

Earlier quoted context omitted.

I do in fact have an algorithm to remove racism from models, but I doubt its worth "$billions". The whole point of my argument is that it shouldn't be necessary. Surely you don't really believe racist stereotypes are true?

I do, in fact. There is extensive research supporting the fact that many (though not all) are accurate. The typical racist stereotype is about twice as likely to replicate as the typical sociology paper. Here are a couple of review articles to get you started: http://www.spsp.org/blog/stereotype-accuracy-response http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... I didn't say that simply removing "racism…

I'm not sure whether or not the stereotype is true. The point is you are forced to acknowledge a pretty unpopular belief to defend this idea. And that the kind of people super concerned about disparate impact are usually not the kind of people that believe that. And I think that if you do acknowledge there are differences, the argument that it's unfair and wrong is much less strong.

In any case the link you posted earlier suggested it could be explained by different rates of numeracy and literacy. Which sound like obvious proxies for IQ. If you gave people an IQ test, or at least literacy or numeracy test, this would probably eliminate any racist bias.

> I said doing so with the same accuracy as models which are "racist" is worth billions.

You necessarily lose some accuracy if you really think race is predictive. But you need not lose all of it. The current methods for removing bias just fuzz features that correlate with race. But there is a way to make sure a model doesn't learn to use features just as a proxy for race, but instead to predict how much they matter independent of race.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#113

Post by Houshalter » Fri, Jul 08, 2016, 1:15 AM UTC

InTheArena wrote: post #108

yummyfajitas wrote: post #89

Earlier quoted context omitted.

As a society we have directed banks to make bad loans to blacks and charge non-blacks extra to make up the difference? I'd be surprised if even 10% of people know this decision was made. Also, what makes it acceptable to engage in this form of surreptitious wealth redistribution on racial lines?

Not being a racist makes it acceptable to not take race or a surrogate for race into condition for a loan ;-) (Please don't take that the wrong way. I am not accusing anyone of racism. Simply stating that at some points our ideals is more valuable then an additional point of profits for the bank). Disparate impact and it's use in credit scoring is mostly governed by Equal Credit Opportunity Act (ECOA), but most of th…

This is only important if you believe that racial stereotypes are true, but should be ignored. That if you control for education, income, region, etc, differences between races are still significant.

Disparate impact goes well beyond removing race as a feature. You sometimes can't use features that correlate with race, even if they are highly predictive. E.g. education.

It also has nothing to do with the profits of banks. Better prediction algorithms for loans mean less interest rates for people that are good borrowers, and less crippling debt to those that aren't. It has huge benefit to the economy and society.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#114

Post by Houshalter » Fri, Jul 08, 2016, 2:01 AM UTC

klipt wrote: post #94

tzs wrote: post #5

The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders that turn down your credit application give you an explanation. From the Federal Trade Commission's site: The creditor must tell you the specific reason for the rejection or that you are entitled to learn the reason if you ask within 60 days. An acceptable reaso…

> The United States does not have anything as sweeping as this The EU takes discrimination much more seriously. For example auto insurance in the EU can't charge men and women different amounts, meanwhile auto insurers in the US shamelessly advertise that they do so.

But men are much riskier drivers. That's has nothing to do with sexism or discrimination against men, it's just an empirical fact. Why should women have to subsidize the cost of men?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#115

Post by yummyfajitas » Fri, Jul 08, 2016, 2:30 AM UTC

InTheArena wrote: post #108

yummyfajitas wrote: post #89

Earlier quoted context omitted.

As a society we have directed banks to make bad loans to blacks and charge non-blacks extra to make up the difference? I'd be surprised if even 10% of people know this decision was made. Also, what makes it acceptable to engage in this form of surreptitious wealth redistribution on racial lines?

Not being a racist makes it acceptable to not take race or a surrogate for race into condition for a loan ;-) (Please don't take that the wrong way. I am not accusing anyone of racism. Simply stating that at some points our ideals is more valuable then an additional point of profits for the bank). Disparate impact and it's use in credit scoring is mostly governed by Equal Credit Opportunity Act (ECOA), but most of th…

So it's racist to act on true beliefs?

Consider a woman walking down the street late at night. Dark alley 1 has a black man. Dark alley 2 has an Asian female. Is she racist if she walks down the alley that minimizes her risk of violence?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#116

Post by dwaltrip » Fri, Jul 08, 2016, 3:18 AM UTC

yummyfajitas wrote: post #115

InTheArena wrote: post #108

Earlier quoted context omitted.

Not being a racist makes it acceptable to not take race or a surrogate for race into condition for a loan ;-) (Please don't take that the wrong way. I am not accusing anyone of racism. Simply stating that at some points our ideals is more valuable then an additional point of profits for the bank). Disparate impact and it's use in credit scoring is mostly governed by Equal Credit Opportunity Act (ECOA), but most of th…

So it's racist to act on true beliefs? Consider a woman walking down the street late at night. Dark alley 1 has a black man. Dark alley 2 has an Asian female. Is she racist if she walks down the alley that minimizes her risk of violence?

Society has chosen to not regulate which alleyways people choose to walk down.

There are many things that individuals are free to do but organizations are not.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#117

Post by yummyfajitas » Fri, Jul 08, 2016, 3:31 AM UTC

dwaltrip wrote: post #116

yummyfajitas wrote: post #115

Earlier quoted context omitted.

So it's racist to act on true beliefs? Consider a woman walking down the street late at night. Dark alley 1 has a black man. Dark alley 2 has an Asian female. Is she racist if she walks down the alley that minimizes her risk of violence?

Society has chosen to not regulate which alleyways people choose to walk down. There are many things that individuals are free to do but organizations are not.

I wasn't asking about regulation, but merely whether this makes the woman racist. Are individuals now unable to be racist?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#118

Post by wodenokoto » Fri, Jul 08, 2016, 8:40 AM UTC

AnthonyMouse wrote: post #106

wodenokoto wrote: post #70

Earlier quoted context omitted.

You need to think about how we measure truth and even what truth is. In machine learning we tend to assume the annotations and labels are "true" and build a system towards that version of the "truth". > Prediction accuracy is measured by whether the predictions are true. The more I think about this sentence, the less sense it makes. Prediction accuracy can only be measured against records of something, and that recor…

> Prediction accuracy can only be measured against records of something, and that record will be a distortion and simplification of reality. Prediction accuracy can be measured against what actually happens . If the algorithm says that 5% of people like Bob will default and you give loans to people like Bob and 7% of them default then the algorithm is off by 2%.

You are still assuming everything that is recorded to be "like Bob" is the truth and captures reality clearly.

Moreover you need to give loans to everybody in order to check the accuracy of the algorithm. You can't just check a non random subset and expect to get non biased results.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#119

Post by AnthonyMouse » Fri, Jul 08, 2016, 1:21 PM UTC

wodenokoto wrote: post #118

AnthonyMouse wrote: post #106

Earlier quoted context omitted.

> Prediction accuracy can only be measured against records of something, and that record will be a distortion and simplification of reality. Prediction accuracy can be measured against what actually happens . If the algorithm says that 5% of people like Bob will default and you give loans to people like Bob and 7% of them default then the algorithm is off by 2%.

You are still assuming everything that is recorded to be "like Bob" is the truth and captures reality clearly. Moreover you need to give loans to everybody in order to check the accuracy of the algorithm. You can't just check a non random subset and expect to get non biased results.

> You are still assuming everything that is recorded to be "like Bob" is the truth and captures reality clearly.

Nope, just finding correlations between "records say Bob has bit 24 set" and "Bob paid his loans." The data could say that Bob is a pink space alien from Andromeda and the algorithm can still do something useful. Because if the data is completely random then it will determine that that field is independent from whether Bob will pay his loans and ignore it, but if it correlates with paying back loans then it has predictive power. The fact that you're really measuring something other than what you thought you were doesn't change that.

> Moreover you need to give loans to everybody in order to check the accuracy of the algorithm. You can't just check a non random subset and expect to get non biased results.

What you can do is give loans to a random subset of the people you wouldn't have to see what happens.

But even that isn't usually necessary, because in reality there isn't a huge cliff right at the point where you decide whether to give the loan or not, and different variables will place on opposite sides of the decision. There will be people you decide to give the loan to even though their income was on the low side because their repayment history was very good. If more of those people than expected repay their loans then you know that repayment history is a stronger predictor than expected and income is a weaker one, and if fewer then the opposite.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#120

Post by klipt » Fri, Jul 08, 2016, 8:55 PM UTC

Houshalter wrote: post #114

klipt wrote: post #94

Earlier quoted context omitted.

> The United States does not have anything as sweeping as this The EU takes discrimination much more seriously. For example auto insurance in the EU can't charge men and women different amounts, meanwhile auto insurers in the US shamelessly advertise that they do so.

But men are much riskier drivers. That's has nothing to do with sexism or discrimination against men, it's just an empirical fact. Why should women have to subsidize the cost of men?

Why is it acceptable to you to have sex explicitly in the model, but not race?

If black men are empirically riskier drivers than Asian men (something insurance companies are currently legally forbidden to discriminate on), would you be fine with charging black men more so Asian men don't "subsidize" them?