Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

breakermag.com

Post reply on HN
Print view View original

181–190 of 388 posts

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#181

Post by unethical_ban »

I would recommend that anyone interested in Machine Learning, or "predictive modeling" etc. should read the book "Weapons of Math Destruction".

https://weaponsofmathdestructionbook.com/

I don't have my notes with me and I'm only 1/3 through, but the main theme is that the best predictive algorithms:

* work transparently for all parties (the creators, users, and "inputs", often people).

* Have no feedback loop (The use of data from the model should not further entrench the output of the model).

And a few others. It gets into discrimination and other major flaws of data modeling re: recidivism, school admissions, stock trading, and other things.

Not all algos are racist - but there are definite attributes to avoid, and this book (or a more rigorous version) should be mandatory reading for all "data scientists".

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#182

Post by ThrustVectoring »

diego898 wrote: post #151
TeMPOraL wrote: post #127

Earlier quoted context omitted.

But I haven't seen cults talking about how to personally ensure you're not falling into trap of cult-like thinking. LW is a rather self-conscious community.

Sorry for the tangent, but from the outside I've also felt very "outside" the LW community. I'm not exactly sure how to "join" it. I know that I'm supposed to read the various required readings first, but then what? How do I see what LW is "talking about" or "fighting about" at a point in time? I mean this in a serious, logistical sense. Thanks!

The LW community has decentralized and deformalized things to a huge extent. It's not just you - the logistics of participating are much less clear than it used to be, and the whole system is far less legible.

Honestly the best strategy is attend local meet-ups, come across as intelligent and interested to the people who hold gatekeeping power over the informal non-public groups, and get in the loop that way. This is significantly easier in Berkeley, where a lot of folks end up moving to. You can also skip some of these steps by attending a CFAR workshop, which will both introduce you to many members of the LW community and put you on an alumni-only mailing list which houses a chunk of the general discussion.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#183

Post by yorwba »

jstarfish wrote: post #84
shadowbained wrote: post #30

Earlier quoted context omitted.

The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide people into groups in order to perform some sort of balancing? Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loa…

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the fi…

> The problem is the self-reinforcing, external factors that these algorithms don't take into account.

And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want.

To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates.

What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse.

In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias.

Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#184

Post by matt9189 »

jpmcglone wrote: post #59

It’s more clear to me than ever now that the left and the right do not use the word “racist” the same way. The left thinks of racism in terms of outcome and the right thinks of racism in terms of intent. We could benefit from better language around these concepts, and honest dialogue about them too.

Broad statements about the left and right are divisive, ignoring the spectrum of beliefs, and this is not a good summary. Just as two examples, housing and education policies are frequently designed with the intent of segregation, though it is impossible to prove intent in most cases. And so, the actual outcomes need to be used in courts to demonstrate racist intent. No one will ever openly admit that they don't want the, for example (as this is common in these cases), poor, mostly black children in their school. But they will also actively ignore all of the research that shows a benefit to their own children with increased diversity. And so intent can be assumed. People will hurt their own families to defend their racist beliefs. Maybe we could use better language, but what is more important is using the language we have more clearly.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#185

Post by 50656E6973 »

hannasanarion wrote: post #172
50656E6973 wrote: post #128

Earlier quoted context omitted.

Statistical differences in outcome cannot be automatically attributed to discrimination, though they frequently are

Yes they can, because that's what "discrimination" means: differences in outcome due to uncontrollable irrelevant factors like race. Discrimination is an effect, not an intent. Racist intent is called "prejudice". Discrimination can be caused by prejudice, or it can be caused by something else, like a poorly considered algorithm.

No, that's pseudo-social-science because it falsely assumes outcomes are solely determined by external variables and is completely ignorant of internal variables, such as differing cultural values.

Some cultures value family life over money, others value money over family life, for example.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#186

Post by viraptor »

nightski wrote: post #105
Nasrudith wrote: post #2

I thought we got over "the computer is never wrong" fallacy by the end of the 90s. Anyone involved with computing should know GIGO - Garbage In Garbage Out.

So you are saying any data for which any input feature is correlated with race or sex is garbage. That's going to make doing useful data science incredibly difficult with anything involving humans.

To some extent. More specifically, if those become confounding variables, the result may be partially useless. That's why you have to make sure you control for race and sex if they're relevant. https://en.m.wikipedia.org/wiki/Confounding

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#187

Post by adlkjnndnnd »

willand31 wrote: post #5

I'm glad to see a representative actually know things like this. Too many of our representatives are technologically illiterate.

Does her tweet really imply any knowledge of it? Maybe she just read in the newspaper about the Google face recognition labeling a black woman as a gorilla?

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#188

Post by ceejayoz »

BeetleB wrote: post #174
ceejayoz wrote: post #167

Earlier quoted context omitted.

There's a third option, though, removing barriers to permit equal access and opportunity . I think it's well illustrated by this comic: https://static1.squarespace.com/static/56d9cbd420c647c7373d4...

That's not a third option - it is the conservative option. If you build a fence that is tall such that some person is disadvantaged, then he doesn't really have the same rights.

The short person still has the right to look over the fence. They just have practical difficulties on exercising it.

Similar cases: Voter ID laws with disproportionate impact on minorities, gay people having "equal" right to marry someone of the opposite gender, people in impoverished school districts having "equal" rights to an education, etc.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#189

Post by 0x8BADF00D »

“Racist” algorithm is a total non-sequitur. This article hilights a disturbing gap in basic math literacy between the normal population and specialized knowledge workers.

Training a neural network on a training set that has primarily “white” faces is no more racist than training a neural network to recognize an orange with a training set primarily made up of blood oranges. I don’t understand the main critique of this article.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#190

Post by hamiltonkibbe »

paublyrne wrote: post #169
paulmd wrote: post #47

Earlier quoted context omitted.

> Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair t…

Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc. Off-topic-ish, but only about 10% of Irish people have red hair.

P(A|B) != P(B|A)
Post reply on HN