11–20 of 87 posts
[flagged]
I won't comment on why it's defined the way that it is.
Edit: it looks like they define several metrics, including ones like I mention above that consider performance and at least one based on what number or percentage is flagged in each group.
Earlier quoted context omitted.
[flagged]
No... the pre-determined bias in this story is obviously that all subgroups of people behave identically w.r.t. welfare applications, which the data itself did not support and a momentary consideration of socioeconomics would debunk. When they tried to cludge the weights to fit their predetermined bias, the model did so poorly on a pilot run that the city shut it down.
From the article:
> Deciding which definition of fairness to optimize for is a question of values and context.
This optimization is the human feedback required to not have the model stagnate in a local optimum.
[flagged]
The better definition of equal performance would obviously be that the metrics for the detector - accuracy or false positive rate etc would be the same for all groups. I won't comment on why it's defined the way that it is. Edit: it looks like they define several metrics, including ones like I mention above that consider performance and at least one based on what number or percentage is flagged in each group.
Key point: The model is considered fair if its performance is equal across these groups. One can immediately see why this is problematic, easily by considering equivalent example in less controversial (i.e. emotionally charged) situations. Should basketball performance be equal across racial, or sex groups? How about marathon performance? It’s not unusual that relevant features are correlated with protected features.…
In your basketball analogy, it's more like they have a model that predicts basketball performance, and they're saying that model should predict performance equally well across groups, not that the groups should themselves perform equally well.
"Unbiased," and "fair" models are generally somewhat ironic. It's generally straightforward to develop one if we don't care much about the performance metric: If we want the output to match a population distribution, we just force it by taking the top predicted for each class and then filling up the class buckets. For example, if we have 75% squares and 25% circles, but circles are predicted at a 10-1 rate, who cares…
As you say, that would be a crappy model. But in my opinion that would also be hardly a fair or unbiased model. That would be a model unfairly biased in favor of HP, who barely sell anything worth recommending
"Unbiased," and "fair" models are generally somewhat ironic. It's generally straightforward to develop one if we don't care much about the performance metric: If we want the output to match a population distribution, we just force it by taking the top predicted for each class and then filling up the class buckets. For example, if we have 75% squares and 25% circles, but circles are predicted at a 10-1 rate, who cares…
So if I want to make a model to recommend inkjet printers then a quarter of all recommendations should be for HP printers? After all, a quarter of all sold printers are HP. As you say, that would be a crappy model. But in my opinion that would also be hardly a fair or unbiased model. That would be a model unfairly biased in favor of HP, who barely sell anything worth recommending
"Unbiased" and "fair" are quite overloaded here, to borrow a programming term.
I think it's one of those times where single words should expressly NOT be used to describe the intent.
The intent of this is to presume that the rate of the thing we are trying to detect is constant across subgroups. The definition of a "good" model therefore is one that approximates this.
I'm curious if their data matches that assumption. Do subgroups submit bad applications at the same rate?
It may be that they don't have the data and therefore can't answer that.
Key point: The model is considered fair if its performance is equal across these groups. One can immediately see why this is problematic, easily by considering equivalent example in less controversial (i.e. emotionally charged) situations. Should basketball performance be equal across racial, or sex groups? How about marathon performance? It’s not unusual that relevant features are correlated with protected features.…
I think they're saying something more subtle. In your basketball analogy, it's more like they have a model that predicts basketball performance, and they're saying that model should predict performance equally well across groups, not that the groups should themselves perform equally well.
The issue is that we don't know how many Danish commit fraud, and we don't know how many Arabs commit fraud, because we don't trust the old process to be unbiased. So how are we supposed to judge if the new model is unbiased? This seems fundamentally impossible without improving our ground truth in some way.
The project presented here instead tries to do some mental gymnastics to define a version of "fair" that doesn't require that better ground truth. They were able to evaluate their results on the false-positive rate by investigating the flagged cases, but they were completely in the dark about the false-negative rate.
In the end, the new model was just as biased, but in the other direction, and performance was simply worse:
> In addition to the reappearance of biases, the model’s performance in the pilot also deteriorated. Crucially, the model was meant to lead to fewer investigations and more rejections. What happened instead was mostly an increase in investigations , while the likelihood to find investigation worthy applications barely changed in comparison to the analogue process. In late November 2023, the city announced that it would shelve the pilot.
Earlier quoted context omitted.
So if I want to make a model to recommend inkjet printers then a quarter of all recommendations should be for HP printers? After all, a quarter of all sold printers are HP. As you say, that would be a crappy model. But in my opinion that would also be hardly a fair or unbiased model. That would be a model unfairly biased in favor of HP, who barely sell anything worth recommending
Yes, well there's the irony. "Unbiased" and "fair" are quite overloaded here, to borrow a programming term. I think it's one of those times where single words should expressly NOT be used to describe the intent. The intent of this is to presume that the rate of the thing we are trying to detect is constant across subgroups. The definition of a "good" model therefore is one that approximates this. I'm curious if their…
Any model would be unfair, age-wise but also ethnically.
To be most effective the model would have to be unfair. It would suck to be a law abiding young specific ethnic minority.
But does it help to search elderly couples?
I’m Genuinely curious what would be fair and effective here. You can’t be a Bayesian.