Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We investigated Amsterdam's attempt to build a 'fair' fraud detection model

lighthousereports.com

Post reply on HN
Print view View original

51–60 of 87 posts

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#51

Post by Borealid »

londons_explore wrote: post #42
tripletao wrote: post #30

Earlier quoted context omitted.

They correctly note the existence of a tradeoff, but I don't find their statement of it very clear. Ideally, a model would be fair in the senses that: 1. In aggregate over any nationality, people face the same probability of a false positive. 2. Two people who are identical except for their nationality face the same probability of a false positive. In general, it's impossible to achieve both properties. If the output…

> Two people who are identical except for their nationality face the same probability of a false positive It would be immoral to disadvantage one nationality over another. But we also cannot disadvantage one age group over another. Or one gender over another. Or one hair colour over another. Or one brand of car over another. So if we update this statement: > Two people who are identical except for any set of properti…

I think the ethical desire is not to remove bias across all properties. Properties that result from an individual's conscious choices are allowed to be used as factors.

One can't change one's race, but changing marital status is possible.

Where it gets tricky is things like physical fitness or social groups...

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#52

Post by precommunicator »

talkingtab wrote: post #24

Is this crazy or what? My take away is that the factors the city of Amsterdam is using to predict fraud are probably not actually predictors. For example if you use the last digit of someones phone number as a fraud predictor, you might discover there is a bias against low numbers. So you adjust your model to make it less likely that low numbers generate investigations. It is unlikely that your model will be any more…

You can find the parameters used in GitHub repository linked from the article, and the phone number isn't one of them (https://github.com/Lighthouse-Reports/amsterdam_fairness/tre...)

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#53

Post by drdaeman »

londons_explore wrote: post #42
tripletao wrote: post #30

Earlier quoted context omitted.

They correctly note the existence of a tradeoff, but I don't find their statement of it very clear. Ideally, a model would be fair in the senses that: 1. In aggregate over any nationality, people face the same probability of a false positive. 2. Two people who are identical except for their nationality face the same probability of a false positive. In general, it's impossible to achieve both properties. If the output…

> Two people who are identical except for their nationality face the same probability of a false positive It would be immoral to disadvantage one nationality over another. But we also cannot disadvantage one age group over another. Or one gender over another. Or one hair colour over another. Or one brand of car over another. So if we update this statement: > Two people who are identical except for any set of properti…

I think you took too much of a jump, considering all properties the same, as if the only way to make the system fair is to make it entirely blind to the applicant.

We tend to distinguish between ascribed and achieved characteristics. It is considered to be unethical to discriminate upon things a person has no control over, such as their nationality, gender, age or natural hair color.

However, things like a car brand are entirely dependent on one's own actions, and if there's a meaningful statistically significant correlation owning a Maserati and fraudulently applying for welfare, I'm not entirely sure it would be unethical to consider such factor.

And it also depends on what a false positive means for a person in question. Fairness (like most things social) is not binary, and while outright rejections can be very unfair, additional scrutiny can be less so, even though still not fair (causing prolonged times and extra stress). If things are working normally, I believe there's a sort of (ever-changing, of course, as times and circumstances evolve) an unspoken social agreement on what's the balance between fairness and abuse that can be afforded.

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#54

Post by bsder »

djohnston wrote: post #4

[flagged]

> Why would you assume that all groupings of people commit welfare fraud at the same rate?

Because the goal is NOT just wiping out fraud, but, instead, minimizing harm or possibly maximizing positive results.

Minimizing fraud is super easy--just don't give out any benefits. No fraud--problem solved.

That's not the final goal, though. As such, the ideal amount of fraud is somewhere above zero. We want to avoid falsely penalizing people who, practically be definition, probably don't have the resources to fight the false classification. And we want to minimize the amount of aid resources we use policing said aid.

The goal is to find a balance. Is helping 100 people but carrying 1 fraudster a good tradeoff? Should it be 1000? Should it be 10? Well, that's a political discussion.

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#55

Post by drdaeman »

dgfitz wrote: post #41
pessimizer wrote: post #33

Earlier quoted context omitted.

> Why would you assume that all groupings of people commit welfare fraud at the same rate? What's the alternative? It's an unattainable statistic, the people who get away with crime. Instead, what ends up getting used is the fraud rates under the old system, or ad hoc rules of thumb based in bigoted anecdotes. So instead you delcare that you don't think that ethnicity is in and of itself a cause of fraud. Even if the…

One of these days, I’m still hopeful, we will figure out that behaviors are taught, usually by parents. Intentionally or accidentally, kids learn from what they see. I don’t care what nationality you are, or what your skin color happens to be, the root cause is how kids are reared. In my head it’s so simple.

Not just kids - it's about one's whole life, including the adulthood, to the very last moments. We tend to change a lot over the courses of our lives, constantly being affected by our surroundings.

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#56

Post by luckylion »

drdaeman wrote: post #53
londons_explore wrote: post #42

Earlier quoted context omitted.

> Two people who are identical except for their nationality face the same probability of a false positive It would be immoral to disadvantage one nationality over another. But we also cannot disadvantage one age group over another. Or one gender over another. Or one hair colour over another. Or one brand of car over another. So if we update this statement: > Two people who are identical except for any set of properti…

I think you took too much of a jump, considering all properties the same, as if the only way to make the system fair is to make it entirely blind to the applicant. We tend to distinguish between ascribed and achieved characteristics. It is considered to be unethical to discriminate upon things a person has no control over, such as their nationality, gender, age or natural hair color. However, things like a car brand…

> It is considered to be unethical to discriminate upon things a person has no control over, such as their nationality, gender, age or natural hair color.

Nationality and natural hair color I understand, but age and gender? A lot of behaviors are not evenly distributed. Riots after a football match? You're unlikely to find a lot of elderly women (and men, but especially women) involved. Someone is fattening a child? That elderly women you've excluded for riots suddenly becomes a prime suspect.

> things like a car brand are entirely dependent on one's own actions

If you assume perfect free will, sure. But do you?

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#57

Post by jaoane »

nxobject wrote: post #50
jaoane wrote: post #45

Earlier quoted context omitted.

[flagged]

> because I don't even need to look at the data to know that some groups are more likely to commit fraud. That is by definition prejudice: bias without evidence. Perhaps they want to avoid that.

Thankfully, this project got evidence. Unfortunately, it was shelved.

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#58

Post by drdaeman »

ordu wrote: post #47
jsemrau wrote: post #36

Earlier quoted context omitted.

>The goal is to avoid penalizing people for their skin color [...] That's not correct. The goal is to identify and flag fraud cases. If one group has a higher likelihood to perform that, then this will show up in the data. The solution should not be to change the data but educate that group to change their behavior. Please note that I have neither mentioned any specific group and do not have a specific group in mind.…

> The solution should not be to change the data but educate that group to change their behavior. 1. This is easier to say than to do. 2. In reality what you see is a correlation. If you try to educate all 20 year old females to not become a connected to organized crime CEOs of construction companies, your efforts will be wasted with 99% of these people, because they are either not connected to organized crime or are…

> The goal is to reduce the amount of fraud cases.

I'm sorry but I fail to see how have you reached this conclusion. Can you please elaborate? The way I understand it, a detection system cannot affect anything about its inputs - you need a feedback loop for that to happen. And I don't see anything like that within the scope of the project as covered by the article.

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#59

Post by drdaeman »

jsemrau wrote: post #48
ordu wrote: post #47

Earlier quoted context omitted.

> The solution should not be to change the data but educate that group to change their behavior. 1. This is easier to say than to do. 2. In reality what you see is a correlation. If you try to educate all 20 year old females to not become a connected to organized crime CEOs of construction companies, your efforts will be wasted with 99% of these people, because they are either not connected to organized crime or are…

"your efforts will lead to a discrimination of 20 years old females" I'd think that this is an extremely far-fetched example that fails at basic logic. Just because a very specific scenario will be flagged does not mean that this scenario is generalized to all CEOs, all females, all 20 year olds.

I think their point is that a lot of us, upon hearing "group A tend to exhibit more of some negative trait X than some other group B" mentally start to associate A with X and this creates a social stigma - just because how our brains work.

I wish there'd be some way to phrase such statements in a nonjudgmental way, without introducing a perception bias...

Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model

#60

Post by Fraterkes »

LorenPechtel wrote: post #49

Why is there so much focus on "fair" even when reality isn't? Not all misdeeds are equally likely to be detected. What matter is minimizing the false positives and false negatives. But it sounds like they don't even have a base truth to be comparing it against, making the whole thing an exercise in bureaucracy.

Who says reality isnt fair? Isnt that up to us, the people inhabiting reality?
Post reply on HN