EU regulations on algorithmic decision-making and a “right to explanation”

I for one of course do not believe that people from other regions/continents are somehow inherently worse or better.

But I do believe that certain elements of different cultures and different ways of social upbringing can have a lasting positive or negative effect on a person. (although who am I to judge what is positive or negative? I try not to do this, I just see differences)

If these things didn't affect how we later as adults view the world, interact with others and respond to different types of challenges in our life then you could expect that basically everyone around the world would have the same moral value system and beliefs about almost everything.

This is obviously not the case.

> We argue that while this law will pose large challenges for industry, it highlights opportunities for machine learning researchers to take the lead in designing algorithms and evaluation frameworks which avoid discrimination.

This sounds like a very slippery slope.

Let me get this straight, what the EU wants to do here. If I make a business that decides how to, say, make loans to people, and I have software that analyzes how they filled out an application or took a written/logical test or something I may have to:

1) Disclose the findings and methodology of my proprietary algorithm

2) Potentially undo decisions that are deemed discriminatory

3) Thereby potentially be forced to choose who I do and do not do business with

Some regulations are good, they help us avoid things like moral hazards. Some regulations, however, act more like power grabs that take away freedom from individuals and businesses.

I'm sorry to say but this sort of thing sounds like the latter.

There are some uncomfortable facts about people and society that algorithms will uncover, and that people will try using to avoid making mistakes. Here's a poignant example, albeit with much less sophistication: http://www.nytimes.com/2015/10/31/nyregion/hudson-city-bank-...

> Instead, some officials say, some banks have quietly institutionalized bias in their operations, deliberately placing branches, brokers and mortgage services outside minority communities, even as other banks find and serve borrowers in those neighbourhoods.

Alright sure. I think we all know it's common sense to not open a jewellery store in inner city Chicago. The local clientele would not be able to afford such products and services, and that's enough of an argument, it's probably business school 101. Just like how you're not going to find a good market for a food delivery startup in Middletown, USA. The economics probably don't work. EDIT - and if you think these businesses are wrong, and they should be in these communities, let their competitors eat their lunch!

But it's viewed as discriminatory. Back to the paper, it's discussing essentially the same thing: it's the natural evolution of "don't start a jewellery store in inner city Chicago" turning into "don't start a jewellery store in neighbourhood where access to urban center is less than X, available nearby 3-lane highways is less than Y, percent of women from 18-29 is less than Z, proportion of people from culture W who are in the bottom quartile of people who buy jewellery is more than V, ...".

The solution here isn't to regulate and force people to open jewellery stores in inner city Chicago. The solution, if we really want to bring diamonds to Englewood, is to drastically reduce the poverty and crime rates so that businesses will want to sell there (https://en.wikipedia.org/wiki/Englewood,_Chicago#Socioeconom...).

And if you think having access to a local business is a "right", then you must concede that such a public good should be provided by a central government of some type. And then you have to ask yourself, do you want the government managing your bank loans, or would you rather have private companies compete on prices and service? If you want the latter, then you need to accept that certain things are just going to arise (or not arise) naturally.

> Prediction accuracy can only be measured against records of something, and that record will be a distortion and simplification of reality.

Prediction accuracy can be measured against what actually happens. If the algorithm says that 5% of people like Bob will default and you give loans to people like Bob and 7% of them default then the algorithm is off by 2%.

Not being a racist makes it acceptable to not take race or a surrogate for race into condition for a loan ;-)

(Please don't take that the wrong way. I am not accusing anyone of racism. Simply stating that at some points our ideals is more valuable then an additional point of profits for the bank).

Disparate impact and it's use in credit scoring is mostly governed by Equal Credit Opportunity Act (ECOA), but most of the banks I am aware of go steps further in ensuring that disparate impact does not occur.

You said: I was with you until that last line. It is not racism to provide better credit conditions to groups that demonstrably have a lower risk of defaulting on a credit.

It absolutely is, under ECOA, which explicitly maintains that you may not discriminate by race for loans.

And btw, as someone very familiar with credit scoring once mentioned to me, a small part of the reason the credit scores do not take into account what you currently make is that ability to pay is often a very poor predictor for willingness to pay. this burned a lot of people in 2006/2007 when it was assumed a good credit score justified a big loan, without any proper verification of income, assets and liabilities.