EU regulations on algorithmic decision-making and a “right to explanation” • Hacker News Board

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#41

Post by Houshalter » Thu, Jul 07, 2016, 2:26 PM UTC

PeterisP wrote: post #8

tzs wrote: post #5

The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders that turn down your credit application give you an explanation. From the Federal Trade Commission's site: The creditor must tell you the specific reason for the rejection or that you are entitled to learn the reason if you ask within 60 days. An acceptable reaso…

A potential problem with this is that if you "tell the applicant they were rejected because their income was too low" then if this is disputed then they will likely be able to point to many other applicants with even lower income that were accepted, because their total combination of factors was better. A simple explanation "factor X is too low" implies that there exists a particular cutoff that is required and suffi…

You can look at the gradient of the inputs. It's not just that your income is too low, but also that you don't have enough credit score, past history, etc. This basically gives you a linear model that is locally accurate at predicting what the more complicated model will do. Linear models are pretty interpretable I think.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#42

Post by JoeAltmaier » Thu, Jul 07, 2016, 2:28 PM UTC

viraptor wrote: post #40

dharma1 wrote: post #25

I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…

Do you know how many actually matter for a single person? I understand that 20k vectors could be created to cover the entire population, but realistically how many are non-zero? I expect there's some threshold below which everything is either completely ignored, or doesn't make a practical difference. An average person's model will contain: - not divorced - not home owner - not prosecuted - not on probation - not dyi…

assuming the vectors are one variable each. Which 'vector' implies they are not. So its more like {particular value of home ownership, age range, medical cost history in a certain bracket, no pet supply purchase history}. How do you explain that? If its been created via data mining and not, for instance, fuzzy logic rule sets, then no human ever knew why each vector was chosen. Its just a pattern discovered in the population.

It would likely be very hard to explain most of the vectors.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#43

Post by JoeAltmaier » Thu, Jul 07, 2016, 2:29 PM UTC

kaijush wrote: post #30

dharma1 wrote: post #25

I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…

>> their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? Well, somehow they decided that their 20k-parameter model is accurate. They should at least be able to explain why they took that decision, even if the model itself is too complex.

Because the training set was covered with x% accuracy. Ok, how does knowing that help?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#44

Post by Houshalter » Thu, Jul 07, 2016, 2:31 PM UTC

tombone12 wrote: post #26

AnthonyMouse wrote: post #19

Earlier quoted context omitted.

> Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Because maximizing prediction accuracy is inherently unbiased. Bias is when the predictions made are inaccurate to the detriment of a group of people. If you had a prediction algorithm that functioned using time trav…

You entirely miss the point! The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.

Thats silly though. You should use ground truth of actual outcomes, not predict that useless humans would do. But even then I bet the algorithm would be less racist than the humans, if you don't give them race as a feature.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#45

Post by jerf » Thu, Jul 07, 2016, 2:33 PM UTC

dublinben wrote: post #31

phkahler wrote: post #24

Earlier quoted context omitted.

>> Finally I get to know why my mortgage was declined :) My first attempt at one was declined due to "problems with my credit report" or some such. I happened to be working at a place where they did credit checks on customers, so I asked someone to pull my credit report. They had all my stuff, but had mixed in a bunch of information from someone else with the same name - different age, there were loans on there from…

This is exactly why you are legally entitled to receive a copy of your credit report every year. You can challenge anything on it, and have incorrect information removed.

When mentioning this, I find it helpful to include this page: https://www.ftc.gov/faq/consumer-protection/get-my-free-cred...

If you are one of the people just finding out about this today, and you Google "free credit report", you come up with a loooot of bad, scammy links. This is how you get your legally-mandated, really free, annual credit report, starting from an ftc.gov address so you know it's really the right one.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#46

Post by hx87 » Thu, Jul 07, 2016, 2:34 PM UTC

tzs wrote: post #5

The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders that turn down your credit application give you an explanation. From the Federal Trade Commission's site: The creditor must tell you the specific reason for the rejection or that you are entitled to learn the reason if you ask within 60 days. An acceptable reaso…

> Then report that the rejection was due to the un-tweaked parameter value.

Shouldn't it be the tweaked parameter value(s), since those are what made the difference between acceptance and rejection?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#47

Post by dharma1 » Thu, Jul 07, 2016, 2:37 PM UTC

kaijush wrote: post #30

dharma1 wrote: post #25

I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…

>> their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? Well, somehow they decided that their 20k-parameter model is accurate. They should at least be able to explain why they took that decision, even if the model itself is too complex.

They validate the accuracy of the model with real users, starting with short term/low value loans, to mitigate risk.

The explanation for them is - it works.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#48

Post by hx87 » Thu, Jul 07, 2016, 2:39 PM UTC

tombone12 wrote: post #26

AnthonyMouse wrote: post #19

Earlier quoted context omitted.

> Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Because maximizing prediction accuracy is inherently unbiased. Bias is when the predictions made are inaccurate to the detriment of a group of people. If you had a prediction algorithm that functioned using time trav…

You entirely miss the point! The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.

Then the problem is in the racist human(s), not the algorithm.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#49

Post by gersh » Thu, Jul 07, 2016, 2:41 PM UTC

Can't you just say the decision was based on a random forest trained on the following parameters with historical data. You could get more specific on how the model was trained or even show model if you have to. I'm sure you can create a small-print tree diagram.

In terms of human review, does this require more than a spot check? I can imagine look over the p-value and possibly spot check the input data, and then approve it.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#50

Post by dilemma » Thu, Jul 07, 2016, 2:41 PM UTC

dharma1 wrote: post #25

I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…

This is exactly why the legislation is created: to stop peoples' lives from being destroyed or negatively affected by computer programs that not even their creator know how they work.