EU regulations on algorithmic decision-making and a “right to explanation”

1. Certain factors are thought to be causally correlated with higher default rates by individuals.

2. Certain populations have higher incidences of individuals matching such factors.

3. These populations have higher than average default rates.

4. Other factors that correlate with membership in these populations will thus be also correlated with higher defalt rates.

5. For an algorithm to be fair, it must be restricted to using only vectors for which causality can be proved.

One of the questions is, even if a vector is "proved" to be causal, would it be disallowed if it also strongly predicted membership in a protected population?

I think you have to be careful there. Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner.

How often are circumstances and context taken into account during sentencing hearings? How often do judges actually try to consider things like whether the offender feels remorse?

Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response?

This is just an example of course, but my point in general is that you make a huge assumption that factors related to decision making can be encoded properly and in an unbiased manner.

I'd even say that many of the factors that would need to be entered into a computer would have to be judged by a human, and therefore leaving the actual decision-making to a computer is a bit of a symbolic gesture in that sense.

I would guess that, given the same data, a computer could/would be programmed to make the same judgement as a human. It is the data extraction process itself that incurs bias, not the algorithm.

Furthermore, if you really leave things up to only externally and easily measurable data, you might not get the results you want or expect. A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? Indeed, a human would have to think really hard to correct this kind of bias. In other words, saying that an algorithm is less biased than a human judge is somewhat of a non-statement, since the judge actually follows fairly strict rules regarding what the facts imply -- it is more how he judges what those facts are that changes the sentencing. Meanwhile, on a computer, the programmer has a lot more power than you might assume in controlling how facts are weighted in order to come up with what is seen as a "fair" decision. And that fairness is a human judgement, fundamentally speaking, since it is about the relationship between a community and an individual.

In short, I don't think you can replace humans with machines when it comes to making decisions about how to deal with humans. The machine can only provide inferences from facts, but the values that those facts imply are a fundamentally human concept. It is a social thing, not an algorithm thing.

Another way to put it: if laws could be perfectly encoded by logic, we would have no need for lawyers and judges. These roles exist because as a society, we are constantly re-evaluating the logic of our society. The rules are not something that can be written statically. They must change as new ideas, problems, and circumstances change.

On a more serious note, I love transparency but again, this is an overeager regulation (not surprising from the EU). You almost never get the true reasons for being rejected, be it in an interview or for a credit, etc. You have to figure that out yourself from the often very vague rejection letter. Our minds are basically running algorithms to which we have even less insight than our computer algorithms. Therefore, this law only hampers the flourishing of the economy while providing no value whatsoever.

Another law that 'solves' a non-issue, brought into being by overpaid career politicians.

Im not assuming that at all. Even with less information , algorithms generally outperform humans. E.g. a hiring algorithm accurately predict job performance from just info on an application, while someone who actually interviews them is barely better than chance. I'm on mobile right now, but I have some links to a lot of research like this, when just one or two simple features is enough to beat human experts.

> Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response?

First of all, the optimal way to do it would be to have the human give a recommendation, then have the algorithm use that as a feature. So it can take human input into account, but also improve on it.

Second, encoding subjective things as real values just lays bare the silliness of it. A human factoring this things in his head is not magically more objective or less biased. Trying to rate subjective features can sometimes improve performance for humans too, since it makes them actually think about it consciously .

> A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think?

But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here.