Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EU regulations on algorithmic decision-making and a “right to explanation”

arxiv.org

Post reply on HN
Print view View original

91–100 of 128 posts

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#91

Post by radarsat1 »

Houshalter wrote: post #79
radarsat1 wrote: post #75

Earlier quoted context omitted.

> Most importantly, most these applications will go back to human judgement. And humans are far worse. I think you have to be careful there. Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. How often are circumstances and context taken into account during sentencing hearings? How often do judges actually try to con…

> Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. Im not assuming that at all. Even with less information , algorithms generally outperform humans. E.g. a hiring algorithm accurately predict job performance from just info on an application, while someone who actually interviews them is barely better than chance. I…

> A human factoring this things in his head is not magically more objective or less biased.

I get your point, and I think you make some good arguments, especially regarding the small number of data points needed. In fact, in this regard I think that machine learning (or statistics anyway) can be a very useful tool for informing decisions. I think it should be regarded similarly to how experts are used in court.

But I think you're missing mine. It is not that humans are better or worse at having bias when considering factors. It is that some factors are inherently biased. And it is the choice of what we do with those biases that we call "values".

Now, do you think "values" are things that can be encoded in a machine-precise fashion?

Well, it's what we call "law". However, law is obviously _not_ something that can be applied "by algorithm," because otherwise we would have nothing to discuss when it comes to hearings.

> But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here.

That is true. I think that categorizing people is something that will never go away. Whether it feels "right" or not is certainly debatable. People with pre-existing conditions aren't exactly happy with the current state of insurance. Do you think black people like it when they can't get a loan because they are black? Do we want society to work that way? You say that the machine can be used to make a better decision -- simple, remove the "skin colour" category. But perhaps that leads to a decrease in the confidence interval. Insurance/banks don't do that, because they consider it important information -- hence, (maybe) the need for regulation. I think this EU regulation is going to be a catalyser for some very interesting discussions around these issues.

Basically what I'm saying is, talking about algorithms vs. humans is a side issue, when the real discussion is, on what kind of data do want decisions to be made?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#92

Post by DominikR »

InTheArena wrote: post #39

Disclaimer - I work in the financial services industry. This is nothing new in most financial service machine learning algorithms have to meet these requirements. This is why why supervised analytics are more popular, and one of the reason why algorithms such as credit scores are typically generated and structured as scorecards with explicit reason codes for each independent characteristic, variable or vector. This i…

> It's my PoV that It's not predictability versus red-tape, and the people trying to do unaccountable analytics in this space are (perhaps in-advertedily) perpetuating racism.

I was with you until that last line. It is not racism to provide better credit conditions to groups that demonstrably have a lower risk of defaulting on a credit.

It would be racism if you offer worse conditions to a certain group without any rational business related explanation.

If some group has a 2x chance on defaulting on a credit then it isn't about the skin of their color - they get to pay more interest because it is more risky to offer this group a credit in the first place.

With that logic you could just as well say that offering better credit conditions to rich people is racist against poor people.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#94

Post by klipt »

tzs wrote: post #5

The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders that turn down your credit application give you an explanation. From the Federal Trade Commission's site: The creditor must tell you the specific reason for the rejection or that you are entitled to learn the reason if you ask within 60 days. An acceptable reaso…

> The United States does not have anything as sweeping as this

The EU takes discrimination much more seriously. For example auto insurance in the EU can't charge men and women different amounts, meanwhile auto insurers in the US shamelessly advertise that they do so.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#96

Post by penguinduck »

sergioisidoro wrote: post #6

I made a couple of experiments on discrimination free machine learning models with naive Bayes, and I changed my perspective on data science. Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Removing sensitive variables (gender, race, etc) doesn't always help, and sp…

You wrote a program that rigs the input data in favor of one class in such a way that the classifier results are more or less uncorrelated with membership in that class. Am I understanding the code correctly?

I don't see how you eliminated "human biases present in the data annotations" here. It seems like your program merely rigs the input in order to get the kind of output you like to see.

Obviously you can tamper with the data to get any kind of result you want. You can force equal outcomes for "Never-worked" and "Private" but that doesn't mean you're removing some inherent bias in the original data.

Also, this sentence:

> All of this excludes the fact that companies will probably maximise to profit, and will use "algorithm" as an excuse to turn down people disregarding ethics.

is kind of weird in the context of your comment and program. If you are correct in your fears of biased data, then accounting for the bias in the data maximizes profit for the companies. Ethics shouldn't be necessary if it's just about bias in the input data.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#97

Post by Houshalter »

yummyfajitas wrote: post #88
Houshalter wrote: post #83

Earlier quoted context omitted.

The bank already knows their assets and income. The question is if an equally poor and educated white is just as likely to repay a loan as an equally poor and educated black. I imagine most of the difference would go away. Unless you really believe black people are inherently less likely to pay back loans, all else equal.

If you can come up with an algorithm that reproduces this conclusion - with accuracy even remotely close to the "racist" ones - banks will cross heaven and earth to pay you $billions. Unfortunately the only public analysis I'm aware of is from a blog using Zillow data: https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t... This effect is reproduced in various walks of life, e.g. education. You'll be doing so…

I do in fact have an algorithm to remove racism from models, but I doubt its worth "$billions". The whole point of my argument is that it shouldn't be necessary. Surely you don't really believe racist stereotypes are true?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#98

Post by dwmtm »

sergioisidoro wrote: post #6

I made a couple of experiments on discrimination free machine learning models with naive Bayes, and I changed my perspective on data science. Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Removing sensitive variables (gender, race, etc) doesn't always help, and sp…

Capitalism leads to some unfavorable outcomes. Algorithms do capitalism better, leading to more unfavorable outcomes. Therefore, algorithms are the problem.

I don't know what I missed, but it looks like algorithms merely provide the reductio for something you already don't like.

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#99

Post by yummyfajitas »

Houshalter wrote: post #97
yummyfajitas wrote: post #88

Earlier quoted context omitted.

If you can come up with an algorithm that reproduces this conclusion - with accuracy even remotely close to the "racist" ones - banks will cross heaven and earth to pay you $billions. Unfortunately the only public analysis I'm aware of is from a blog using Zillow data: https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t... This effect is reproduced in various walks of life, e.g. education. You'll be doing so…

I do in fact have an algorithm to remove racism from models, but I doubt its worth "$billions". The whole point of my argument is that it shouldn't be necessary. Surely you don't really believe racist stereotypes are true?

I do, in fact. There is extensive research supporting the fact that many (though not all) are accurate. The typical racist stereotype is about twice as likely to replicate as the typical sociology paper.

Here are a couple of review articles to get you started: http://www.spsp.org/blog/stereotype-accuracy-response http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-...

I didn't say that simply removing "racism" (by which I assume you mean disparate impact) is worth billions. I said doing so with the same accuracy as models which are "racist" is worth billions. Obviously you can do anything you want if you give up accuracy.

Why do you believe they are false? Simply because it's socially unacceptable to think otherwise?

Re: EU regulations on algorithmic decision-making and a “right to explanation”

#100

Post by zeveb »

sergioisidoro wrote: post #6

I made a couple of experiments on discrimination free machine learning models with naive Bayes, and I changed my perspective on data science. Usually people are concerned about maximising prediction accuracy, and never stop to think about what correlations is the model finding down below, and the human biases present in the data annotations. Removing sensitive variables (gender, race, etc) doesn't always help, and sp…

> All of this excludes the fact that companies will probably maximise to profit, and will use "algorithm" as an excuse to turn down people disregarding ethics.

If companies are maximising profit, then they will treat a person _exactly_ according to everything they can observe about that person. How is that unethical?

The problem that people have is that they don't want objectivity: they want to be able to pretend that something false is true, because it is comfortable.

Post reply on HN