We investigated Amsterdam's attempt to build a 'fair' fraud detection model
lighthousereports.com
We investigated Amsterdam's attempt to build a 'fair' fraud detection model
1–10 of 87 posts
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#2"Unbiased," and "fair" models are generally somewhat ironic.
It's generally straightforward to develop one if we don't care much about the performance metric:
If we want the output to match a population distribution, we just force it by taking the top predicted for each class and then filling up the class buckets.
For example, if we have 75% squares and 25% circles, but circles are predicted at a 10-1 rate, who cares, just take the top 3 squares predicted and the top 1 circle predicted until we fill the quota.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#3"Unbiased," and "fair" models are generally somewhat ironic. It's generally straightforward to develop one if we don't care much about the performance metric: If we want the output to match a population distribution, we just force it by taking the top predicted for each class and then filling up the class buckets. For example, if we have 75% squares and 25% circles, but circles are predicted at a 10-1 rate, who cares…
But that's a bias, if circles are actually more likely to be fraudulant.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#4[flagged]
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#5The article talks a lot about fairness metrics but never mentions whether the system actually catches fraud.
Without figures for true positives, recall, or financial recoveries, its effectiveness remains completely in the dark.
In short: great for moral grandstanding in the comments section, but zero evidence that taxpayer money or investigative time was ever saved.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#6[flagged]
Yes it is. This is some ideal world thinking, that has nothing to do with reality and is easily falsifiable, but only if you want to see the real world.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#7[flagged]
[flagged]
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#8"Unbiased," and "fair" models are generally somewhat ironic. It's generally straightforward to develop one if we don't care much about the performance metric: If we want the output to match a population distribution, we just force it by taking the top predicted for each class and then filling up the class buckets. For example, if we have 75% squares and 25% circles, but circles are predicted at a 10-1 rate, who cares…
But that's a bias, if circles are actually more likely to be fraudulant.
If the definition of "unbiased" and "fair" is that the model flags squares and circles at a rate or proportion equal to the population distribution of squares and circles, then the model is unbiased and fair.
As noted above, this doesn't do anything for performance.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#9[flagged]
[flagged]
No... the pre-determined bias in this story is obviously that all subgroups of people behave identically w.r.t. welfare applications, which the data itself did not support and a momentary consideration of socioeconomics would debunk. When they tried to cludge the weights to fit their predetermined bias, the model did so poorly on a pilot run that the city shut it down.
Re: We investigated Amsterdam's attempt to build a 'fair' fraud detection model
#10Key point:
The model is considered fair if its performance is equal across these groups.
One can immediately see why this is problematic, easily by considering equivalent example in less controversial (i.e. emotionally charged) situations.
Should basketball performance be equal across racial, or sex groups? How about marathon performance?
It’s not unusual that relevant features are correlated with protected features. In the specific example above, being an immigrant is likely correlated with not knowing the local language, therefore being underemployed and hence more likely to apply for benefits.