Finally I get to know why my mortgage was declined :)
>> Finally I get to know why my mortgage was declined :) My first attempt at one was declined due to "problems with my credit report" or some such. I happened to be working at a place where they did credit checks on customers, so I asked someone to pull my credit report. They had all my stuff, but had mixed in a bunch of information from someone else with the same name - different age, there were loans on there from…
EU regulations on algorithmic decision-making and a “right to explanation”
31–40 of 128 posts
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#32I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…
What your example implies is that there are business that doesn't take discrimination laws into account when designing their software, even for sensitive decisions. And enforcing discrimination free decisions is exactly the goal of this law.
I worked for the gambling industry. "changing our software to meet the law is too complicated" never was a valid reasoning, even when complex and extremely hard to fine tune algorithms were at stake.
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#33I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#34I think this is really bad. That's the majority of uses of machine learning. It also has a lot of economic value to predict things like how likely someone is to pay back a loan, or even who is a spammer.
Most importantly, most these applications will go back to human judgement. And humans are far worse. Human predictions about candidates are really bad. We are incredibly biased by things like race, gender, and especially attractiveness. Unattractive people get twice the sentences of attractive people. Not to mention random stuff, like people being judged more harshly when the judge is hungry before lunch. They also rarely give explanations for their decisions, if they are even aware of the true reasons for them.
Going back to humans is a huge step backwards and will hurt a lot more people than it helps. I think the same regulations that apply to algorithms, should apply to humans. That would show the absurdity of these laws. But humans are algorithms after all. And particularly bad ones (for this purpose anyway.)
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#35Re: EU regulations on algorithmic decision-making and a “right to explanation”
#36Earlier quoted context omitted.
>> Finally I get to know why my mortgage was declined :) My first attempt at one was declined due to "problems with my credit report" or some such. I happened to be working at a place where they did credit checks on customers, so I asked someone to pull my credit report. They had all my stuff, but had mixed in a bunch of information from someone else with the same name - different age, there were loans on there from…
This is exactly why you are legally entitled to receive a copy of your credit report every year. You can challenge anything on it, and have incorrect information removed.
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#37Earlier quoted context omitted.
> Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Because maximizing prediction accuracy is inherently unbiased. Bias is when the predictions made are inaccurate to the detriment of a group of people. If you had a prediction algorithm that functioned using time trav…
You entirely miss the point! The point is that in supervised learning for example, if you optimize prediction accuracy with respect to your human generated examples, you will get a model that exactly reproduces the racist judgment of the human that generated your training set.
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#38I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…
> I think it will be very hard to implement/enforce this regulation. The regulation is quite simple and easy to enforce. If they can't explain then it is illegal. What your example implies is that there are business that doesn't take discrimination laws into account when designing their software, even for sensitive decisions. And enforcing discrimination free decisions is exactly the goal of this law. I worked for th…
I never thought I'd say it, but relatively speaking, the gambling industry is grounded in reality.
Most of the commenters here are coming from Silicon Valley where regulation is virtually non-existent and typically derided as impeding the glorious all-consuming will of the free market.
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#39This is nothing new in most financial service machine learning algorithms have to meet these requirements. This is why why supervised analytics are more popular, and one of the reason why algorithms such as credit scores are typically generated and structured as scorecards with explicit reason codes for each independent characteristic, variable or vector.
This is really needed for a number of different reasons. The biggest is that algorithms are increasingly running our world. Even outside of mortgage, have you ever had to fork over the right to pull your credit score for either rent or (in some limited cases) to apply for a job? Not being able to understand: a) Why you were rejected. b) Why you are paying a higher interest rate. c) What you can do to fix that?
Housing and employment is fundamentally a question of human rights, not just banking profitability or academic algorithm improvements.
Part of the reason why Credit Scoring (which is intrinsically algorithmic decision-making) is because it replaced the "old boys" network that used to dominate. In this world you went to your banker, and the decision to extend credit was dominated by your personal relationship with a banker who might not be the same ethnicity, religion, nationality, etc as you would dominate your ability to purchase housing. The credit score democratized financial access.
It can still be used to discriminate. There have been a number of studies that have shown that simply being from a white community tends to increase the number of other people you can lean on in a emergency, and hence make you less likely to default on a loan. From a pure predictability point of view, a bank is at a lower risk of default lending to someone from that community, but that in turn denies financial access to people not in that community, continuing the trend.
A big problem in this kind of financial model is that it's relatively easy to accidentally or deliberately find other variables that mirror "protected" vectors. A simplistic example is zip code, where the zip code reflects a area that is predominately ethnic minorities.
So it's not cut and dried. It's my PoV that It's not predictability versus red-tape, and the people trying to do unaccountable analytics in this space are (perhaps in-advertedily) perpetuating racism.
Re: EU regulations on algorithmic decision-making and a “right to explanation”
#40I think it will be very hard to implement/enforce this regulation. I attended the London AI summit last week, and they had a speaker from a German lender called Kreditech. There was a question of black box credit scoring, and the speaker made a fair point - their models have 20,000 vectors in determining credit worthiness. How would you begin to break that down to something explainable? You can list the sources of th…
I expect there's some threshold below which everything is either completely ignored, or doesn't make a practical difference. An average person's model will contain:
- not divorced
- not home owner
- not prosecuted
- not on probation
- not dying
- not (19k of situations)