Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep-Fried Data

idlewords.com

Post reply on HN
Print view View original

131–140 of 149 posts

Re: Deep-Fried Data

#131

Post by AndrewKemendo »

coldtea wrote: post #87
AndrewKemendo wrote: post #80

Earlier quoted context omitted.

Individual (or aggregate) human researchers are not hooked up in huge services to make inferences and deductions automatically about billions of people Yes they (we) are. It's the same data set. TV, movies, papers, internet videos et al. is all the same biased, labeled data that is being fed (watched, listened to etc...) to machines. You automatically make inferences and deduce things about people based on labeling a…

> Yes they (we) are. I didn't say researchers and/or individual people are not making such judgements about billions of others (e.g. "the Chinese suck/are great"). I said they are not "hooked up in huge services" to make them automatically for billions of individuals -- like an ML algorithm used by Google or Amazon or some government agency etc would do.

My point is that it's the same thing. Individuals with outsized power and influence can affect billions of people based on their own judgments and implicitly make those judgments automatically for billions.

That's what this question is all about - should machine systems be responsible for the kind of sweeping decisions that humans are making on those populations now? Probably.

Re: Deep-Fried Data

#132

Post by mrow84 »

yummyfajitas wrote: post #119
mrow84 wrote: post #118

Earlier quoted context omitted.

I don't really understand that claim. You are explicitly adding a bias that is linearly dependant on your race variable, and then allowing your regression to recover that bias by introducing noisy measurements of race (which you as the modeller knew was the thing causing the bias). As you say, that is unsurprising. That result does not, however, address my point, which is that if the structure of the bias is difficul…

The point is that the bias is linear, and my model is linear, so the model fixes things. The example is synthetic (so we could know what the right answer is and check if we recover it) so of course I put everything in. In the linked article, I explicitly reference a real world case where the same linear model was used to discover that grades and test scores are biased in favor of blacks: http://ftp.iza.org/dp8733.pdf…

To remain with the example in your blog post, your model fixed things because the implicit bias model was correct (linear dependance on race), and the data were available, either directly (via the race variable) in the "What if measurements are biased?" section, or indirectly (via the noisy redundantly-encoded race variables) in the "What if we scrub race, but redundantly encode it?" section.

In the first of those two sections you yourself note how bias correction is not possible without the relevant data: "If we scrubbed the data this result would be impossible. Running least squares on scrubbed data yields alpha = [ 0.29878373, 0.30869833] - we can't correct for bias because we don't know the variable being biased on."

I'm not disputing that bias correction is possible, only that it can be much harder than you seem to be implying, with statements like "Most algorithms can and will correct for biases in their inputs.", and "Of course data contains biases. But again, please read the article I linked; algorithms will have a tendency to correct that bias."

I have some experience with bias correction in (ocean) weather forecasting, and in that domain there were problems both with the difficulty of modelling the bias structure, and with obtaining measurements reliable enough for bias correction.

Re: Deep-Fried Data

#133

Post by yummyfajitas »

eridius wrote: post #126
yummyfajitas wrote: post #125

Earlier quoted context omitted.

I directly addressed this critique two posts up. Why don't you go read that post? https://news.ycombinator.com/item?id=12627359

I did read it, but you're talking about correcting for measurement biases in order to recover an accurate view of reality. But what I'm saying is that accurately measuring reality may in fact be how you get bias, because the very thing you're measuring may be biased. If you're aware the bias exists and have tools that can measure the bias itself then maybe you can correct for the bias, but you can't just expect your…

Here's my concrete claim. Let pp = police presence, then P(crime detected) = r(pp).

Measured crime = crimes x r(police presence).

As long as your model is expressive enough to capture r(pp), bias should be detected.

Fundamentally you are making the claim that there are certain types of variable correlations that are just so evil that no statistical model can possibly understand them. That's a very bold claim; it's essentially the claim that science doesn't work.

Re: Deep-Fried Data

#134

Post by yummyfajitas »

mrow84 wrote: post #132
yummyfajitas wrote: post #119

Earlier quoted context omitted.

The point is that the bias is linear, and my model is linear, so the model fixes things. The example is synthetic (so we could know what the right answer is and check if we recover it) so of course I put everything in. In the linked article, I explicitly reference a real world case where the same linear model was used to discover that grades and test scores are biased in favor of blacks: http://ftp.iza.org/dp8733.pdf…

To remain with the example in your blog post, your model fixed things because the implicit bias model was correct (linear dependance on race), and the data were available, either directly (via the race variable) in the "What if measurements are biased?" section, or indirectly (via the noisy redundantly-encoded race variables) in the "What if we scrub race, but redundantly encode it?" section. In the first of those tw…

What I'm disputing is this:

Machine learning does not have less bias than human researchers. It is simply magnified at scale.

This is fundamentally wrong. Given data on the biasing factor, most algorithms will try to use it and improve things. Sometimes information is unavailable. On net there is a reason why many algorithms will reduce bias, and no particular reason why they would increase it equally in the remaining cases.

Re: Deep-Fried Data

#135

Post by dragonwriter »

yummyfajitas wrote: post #134
mrow84 wrote: post #132

Earlier quoted context omitted.

To remain with the example in your blog post, your model fixed things because the implicit bias model was correct (linear dependance on race), and the data were available, either directly (via the race variable) in the "What if measurements are biased?" section, or indirectly (via the noisy redundantly-encoded race variables) in the "What if we scrub race, but redundantly encode it?" section. In the first of those tw…

What I'm disputing is this: Machine learning does not have less bias than human researchers. It is simply magnified at scale. This is fundamentally wrong. Given data on the biasing factor, most algorithms will try to use it and improve things. Sometimes information is unavailable. On net there is a reason why many algorithms will reduce bias, and no particular reason why they would increase it equally in the remainin…

> Given data on the biasing factor, most algorithms will try to use it and improve things.

Unless the bias is in what they are designed to optimize for (either because the goal is explicitly biased or because the operationalization of the goal into a concrete measure is, whether intentionally or not, biased), in which case they will obviously reinforce it.

Re: Deep-Fried Data

#136

Post by Natanael_L »

wtracy wrote: post #51
keyle wrote: post #44

Earlier quoted context omitted.

Same here. I'm battling with this thought a lot. Beyond jobs, I think there should be communities of developers, designers, producers, writers, getting together and figuring out this stuff. And I don't mean open source projects. Let's group together smart people wanting to make a difference and have a hit list of things we (people) actually need. A group that would organise people into mission driven development. I'm…

> And I don't mean open source projects. Then what do you mean? You described exactly what some of the largest, most successful FOSS projects (Firefox, KDE, Gnome, Libre Office, FreeBSD) are already doing. > Let's group together smart people wanting to make a difference and have a hit list of things we (people) actually need. Well, the FSF maintains a list of "high priority Free Software projects" that need help, but…

Sounds like his thought is more about building infrastructure, and even non-technical solutions (as in rethinking policy, scientific research projects, etc).

Like a global collaborative project to improve just about everything.

Re: Deep-Fried Data

#137

Post by eridius »

yummyfajitas wrote: post #133
eridius wrote: post #126

Earlier quoted context omitted.

I did read it, but you're talking about correcting for measurement biases in order to recover an accurate view of reality. But what I'm saying is that accurately measuring reality may in fact be how you get bias, because the very thing you're measuring may be biased. If you're aware the bias exists and have tools that can measure the bias itself then maybe you can correct for the bias, but you can't just expect your…

Here's my concrete claim. Let pp = police presence, then P(crime detected) = r(pp). Measured crime = crimes x r(police presence). As long as your model is expressive enough to capture r(pp), bias should be detected. Fundamentally you are making the claim that there are certain types of variable correlations that are just so evil that no statistical model can possibly understand them. That's a very bold claim; it's es…

No, I'm claiming that P(crime detected) != r(pp). More police in an area typically means more crime is detected, but that's not the only factor. If you have two areas with identical police presences and identical actual crime rates (as opposed to reported crime rates), the rate of crime detection (as measured by arrests and whatnot) may be higher in one area due to other factors such as racial bias (not just racial profiling, but also things like police letting white people off with a warning where the equivalent black person would be arrested). So you cannot simply correct for this by accounting for the police presence.

What's more, your data may not even have the necessary info to figure out if there's a bias. For example, what if police are more likely to arrest someone wearing a red shirt than someone wearing any other color shirt? Unless the color of the person's shirt is part of the arrest report, there's no way your statistical model is going to figure out that red shirts affect arrest rate.

Re: Deep-Fried Data

#138

Post by yummyfajitas »

eridius wrote: post #137
yummyfajitas wrote: post #133

Earlier quoted context omitted.

Here's my concrete claim. Let pp = police presence, then P(crime detected) = r(pp). Measured crime = crimes x r(police presence). As long as your model is expressive enough to capture r(pp), bias should be detected. Fundamentally you are making the claim that there are certain types of variable correlations that are just so evil that no statistical model can possibly understand them. That's a very bold claim; it's es…

No, I'm claiming that P(crime detected) != r(pp). More police in an area typically means more crime is detected, but that's not the only factor. If you have two areas with identical police presences and identical actual crime rates (as opposed to reported crime rates), the rate of crime detection (as measured by arrests and whatnot) may be higher in one area due to other factors such as racial bias (not just racial p…

Your function r = r(pp, red shirts, race of offender, etc) exists. A model of the form a x r + b x something_else + ... will detect the bias you've described, assuming of course the biasing variable is either present or redundantly encoded in the data set.

We've now established the existence of a statistical model which can detect this bias.

Now, any other model which is capable of expressing your specific r(p) can do the same thing. The entire purpose of fancy models like random forests is that they can express lots of functions while also being reasonably generalizable.

If you want to claim that this bias is much more difficult to encode in an SVM than all the other typical hidden patterns, you need to establish that your specific r(...) is somehow vastly more complicated than all the other things that machine learning models regularly detect. That's a pretty strong claim.

Interestingly, you are now arguing the exact opposite of what most "machine learning is racist" people claim. They typically claim machine learning is racist because algorithms actually learn hidden factors they wish it wouldn't; e.g., a lending algorithm might "redline" blacks who don't pay back their debts. I take it you believe this is highly unlikely, and algorithms can't possibly distinguish between men and women and then show high paying job ads to more men than women?

Re: Deep-Fried Data

#139

Post by eridius »

yummyfajitas wrote: post #138
eridius wrote: post #137

Earlier quoted context omitted.

No, I'm claiming that P(crime detected) != r(pp). More police in an area typically means more crime is detected, but that's not the only factor. If you have two areas with identical police presences and identical actual crime rates (as opposed to reported crime rates), the rate of crime detection (as measured by arrests and whatnot) may be higher in one area due to other factors such as racial bias (not just racial p…

Your function r = r(pp, red shirts, race of offender, etc) exists. A model of the form a x r + b x something_else + ... will detect the bias you've described, assuming of course the biasing variable is either present or redundantly encoded in the data set. We've now established the existence of a statistical model which can detect this bias. Now, any other model which is capable of expressing your specific r(p) can d…

Regarding your last paragraph, there's two different angles here. The "machine learning is racist" angle I think is quite valid, but covers a different topic than what we've been discussing here. To be more specific, there's two different ways in which we can have racist models:

1. The algorithm is biased in a way that reflects reality but does not reflect how we wish it to behave. This is the "machine learning is racist" angle. A lending algorithm might quite rightly think that black people are a higher risk, but this is ethically problematic to act on, because denying loans to black people only serves to compound the social problem (even though it may make financial sense for your bank).

2. What I'm arguing is that we can have racist algorithms due to the fact that the data itself may be biased in a way you're not aware of. To take the red shirt example, something I forgot to say before was that if, say, a fad spreads among the black community of wearing red shirts, then you're going to see an uptick in arrests of black people, but your algorithm won't be able to figure out that this is actually due to arresting red-shirted people, which means it will believe that black people in general are more likely to be arrested.

Re: Deep-Fried Data

#140

Post by wrsh07 »

dragonwriter wrote: post #124
eridius wrote: post #123

Earlier quoted context omitted.

The problem is you're modeling a biased reality. And accurately modeling a biased reality may in many cases accentuate the bias. Take for example the previously-mentioned case of using an algorithm to determine where to focus your policing efforts. If the data you have says that more arrests are done in a particular part of the city, then you'll want to put more police there, right? But areas where there are more pol…

The problems are really twofold: (1) Defining the proper goals, and (2) Measuring the right things (such as the real goals of interest rather than biased proxies.) With police deployments, you are assuming the solution (rather than letting your algorithm optimize it) by saying "I want to put more police where more arrests occur". What you really want is probably something more like (the exact goal may be different, o…

Thank you. This is so much clearer than what I was saying.

As you say, proper goals and measurement can fix a lot of these problems, and I don't think it's obvious that ml algorithms solve either of those

Post reply on HN