EU regulations on algorithmic decision-making and a “right to explanation” • Hacker News Board

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#51

Post by germanier » Thu, Jul 07, 2016, 2:47 PM UTC

Houshalter wrote: post #34

Is my reading of this law correct? It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions where a "member state" authorizes it. But the legalese is difficult to parse, and no one else seems to get this impression. I think this is really bad. That's the majority of uses of machine learning. It also has a lot of economic value to predict things like…

> It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions where a "member state" authorizes it.

Who says that this is rare? That's a pretty standard way EU regulations are written to mean "Action A can only be allowed if conditions B are met".

If the regulation is approved, member states take a look and change their laws accordingly. They could outlaw algorithmic decision making if they wanted to (but they wouldn't need that regulation to do that). Much more likely is that they impose those conditions on the companies that use such algorithms just like they already impose a lot of conditions (such as "ethnicity is not allowed to be a direct input variable" currently is).

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#52

Post by AnthonyMouse » Thu, Jul 07, 2016, 2:49 PM UTC

tombone12 wrote: post #26

AnthonyMouse wrote: post #19

Earlier quoted context omitted.

> Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Because maximizing prediction accuracy is inherently unbiased. Bias is when the predictions made are inaccurate to the detriment of a group of people. If you had a prediction algorithm that functioned using time trav…

You entirely miss the point! The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.

> The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.

In which case you aren't optimizing prediction accuracy. Prediction accuracy is measured by whether the predictions are true. If you have bias in the predictions which doesn't exist in the actual outcomes then there is money to be made by eliminating it.

It seems like the strangest place to put an objection where the profit motive is directly aligned with the desired behavior.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#53

Post by sapphireblue » Thu, Jul 07, 2016, 2:50 PM UTC

throwawaysocks wrote: post #21

crististm wrote: post #12

Earlier quoted context omitted.

You are certainly right to question the reasoning behind AI decisions. I'm not sure that neural network based AI can do that though. For reference see an older thread on this very topic: https://news.ycombinator.com/item?id=10388795

> I'm not sure that neural network based AI can do that though. Generating human-readable explanations from a machine learning algorithm or NN is probably an impossible problem to solve in full generality. However, for many particular systems, doing so should be feasible.

Explaining how neural network models generate their output is possible, see http://cs231n.github.io/understanding-cnn/ https://auduno.github.io/2016/06/18/peeking-inside-convnets/ https://courses.cs.washington.edu/courses/cse590v/14au/cse59...

Also it should be possible to train the NN model to explain its decision, even in plain language.

I think the "right to explanation" law is pretty sensible, people should have a right to question the algorithmic judgment that impacts their life. It will be somewhat inconvenient for big companies applying ML to make these decisions, but it won't ruin their business.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#54

Post by zoner » Thu, Jul 07, 2016, 2:51 PM UTC

dublinben wrote: post #31

phkahler wrote: post #24

Earlier quoted context omitted.

>> Finally I get to know why my mortgage was declined :) My first attempt at one was declined due to "problems with my credit report" or some such. I happened to be working at a place where they did credit checks on customers, so I asked someone to pull my credit report. They had all my stuff, but had mixed in a bunch of information from someone else with the same name - different age, there were loans on there from…

This is exactly why you are legally entitled to receive a copy of your credit report every year. You can challenge anything on it, and have incorrect information removed.

Not in the EU (or at least not in The UK where I currently live).

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#55

Post by yoo1I » Thu, Jul 07, 2016, 2:53 PM UTC

hx87 wrote: post #48

tombone12 wrote: post #26

Earlier quoted context omitted.

You entirely miss the point! The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.

Then the problem is in the racist human(s), not the algorithm.

But the algorithm enshrines and possibly amplifies it. Or as the old saying goes:

> To err is human, to really foul things up requires a computer.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#56

Post by sshumaker » Thu, Jul 07, 2016, 3:00 PM UTC

jerf wrote: post #45

dublinben wrote: post #31

Earlier quoted context omitted.

This is exactly why you are legally entitled to receive a copy of your credit report every year. You can challenge anything on it, and have incorrect information removed.

When mentioning this, I find it helpful to include this page: https://www.ftc.gov/faq/consumer-protection/get-my-free-cred... If you are one of the people just finding out about this today, and you Google "free credit report", you come up with a loooot of bad, scammy links. This is how you get your legally-mandated, really free, annual credit report, starting from an ftc.gov address so you know it's really the right…

Credit Karma is legit and free. Also gives you your credit score and credit monitoring (all free).

(Disclaimer: I work there).

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#57

Post by [deleted] » Thu, Jul 07, 2016, 3:00 PM UTC

jerf wrote: post #45

dublinben wrote: post #31

Earlier quoted context omitted.

This is exactly why you are legally entitled to receive a copy of your credit report every year. You can challenge anything on it, and have incorrect information removed.

When mentioning this, I find it helpful to include this page: https://www.ftc.gov/faq/consumer-protection/get-my-free-cred... If you are one of the people just finding out about this today, and you Google "free credit report", you come up with a loooot of bad, scammy links. This is how you get your legally-mandated, really free, annual credit report, starting from an ftc.gov address so you know it's really the right…

[deleted]

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#58

Post by [deleted] » Thu, Jul 07, 2016, 3:00 PM UTC

InTheArena wrote: post #39

Disclaimer - I work in the financial services industry. This is nothing new in most financial service machine learning algorithms have to meet these requirements. This is why why supervised analytics are more popular, and one of the reason why algorithms such as credit scores are typically generated and structured as scorecards with explicit reason codes for each independent characteristic, variable or vector. This i…

[deleted]

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#59

Post by sergioisidoro » Thu, Jul 07, 2016, 3:01 PM UTC

InTheArena wrote: post #39

Disclaimer - I work in the financial services industry. This is nothing new in most financial service machine learning algorithms have to meet these requirements. This is why why supervised analytics are more popular, and one of the reason why algorithms such as credit scores are typically generated and structured as scorecards with explicit reason codes for each independent characteristic, variable or vector. This i…

I think the case of zip code based discrimination is even named. It's called redlining, and is being used in machine learning world to describe indirect discrimination based on certain attributes (eg. discriminate people based on their zip code, a zip code with mostly black population for example)

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#60

Post by Kristine1975 » Thu, Jul 07, 2016, 3:21 PM UTC

mknocker wrote: post #10

The first thing that came to my mind was... what about advertising on the web ? This is an area where machine learning and algorithmic decision-making use discrimination to either show or not an ad to a particular user. There are millions decision made everyday. Under that regulation, could you ask for the reason an ad was shown to you ?

Just put a link under the ad "why am I seeing this ad?" leading to a page with the explanation.