Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

breakermag.com

Post reply on HN
Print view View original

131–140 of 388 posts

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#131

Post by hannasanarion »

mehrdadn wrote: post #14
kauffj wrote: post #11

There is no design of AI that can be simultaneously utilitarian, procedurally fair, and representatively fair. I'm going to repeat this again, since many people struggle with this. It is _literally impossible_ to achieve the best utilitarian outcome, the most procedurally fair outcome, and a representatively fair outcome. Anyone designing algorithms will have to make tradeoffs along this frontier. For a far better ar…

This sounds an awful lot like voting impossibility theorems (Arrow's and its generalizations)... are they related?

More like the orthogonality thesis, I think. These different value axes are totally unrelated, and building an algorithm that maximizes one does not necessarily mean that it maximizes the others as well, and this mismatch can have disastrous consequences, a la "The Paperclip Machine"

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#132

Post by duskwuff »

saalweachter wrote: post #124
tunesmith wrote: post #66

Earlier quoted context omitted.

Those words are "systemic" or "implicit" or "institutional", etc. All you really need is a proof or example that racist/sexist outcomes are possible even when there is no overt intent. And there are plenty of examples like that out there. Failure to accept that those examples exist, however, is something beyond just looking at intent instead of outcome.

I'd also like to bring up the International Obfuscated C Code Contest. Just because a policy seems reasonable and has straightforward justifications for all of its pieces doesn't mean it wasn't maliciously designed to another purpose. The stated intent is not always the only intent and if the results...

Perhaps even more apropos to your point is the Underhanded C Contest:

http://www.underhanded-c.org/

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#133

Post by tunesmith »

saalweachter wrote: post #124
tunesmith wrote: post #66

Earlier quoted context omitted.

Those words are "systemic" or "implicit" or "institutional", etc. All you really need is a proof or example that racist/sexist outcomes are possible even when there is no overt intent. And there are plenty of examples like that out there. Failure to accept that those examples exist, however, is something beyond just looking at intent instead of outcome.

I'd also like to bring up the International Obfuscated C Code Contest. Just because a policy seems reasonable and has straightforward justifications for all of its pieces doesn't mean it wasn't maliciously designed to another purpose. The stated intent is not always the only intent and if the results...

That's true too, yes. I just think it's interesting that malicious intent isn't required. I think it helps to talk about it too - if you can approach the participants and assume good faith, allow them to believe that you believe the output is completely unintentional, then maybe it will be easier for them to agree to change their system.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#134

Post by cmiles74 »

blfr wrote: post #113
cmiles74 wrote: post #82

Earlier quoted context omitted.

Thinking of racism in terms of the intent is a way to take the issue less seriously, to force the issue into the realm of opinions and beliefs. If the intent of the perpetrator is the deciding factor, then it is always arguable that perpetrator's intent was not specifically racist; perhaps it was driven by a misunderstanding, etc. Since it was informed by misunderstanding, it isn't racist. And so on. It's an old tact…

But racism is from the realm of opinions and beliefs.

A lot of people rationalize racist or discriminatory beliefs in such a way that they seem reasonable. Yet, when you judge their behavior as a whole, it tells a different story. The banker who consistently gives loans to people from [group x] but not from [group y] who have similar financial features. When we talk to the hypothetical banker, they may claim that "numbers don't tell the whole story", etc.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#135

Post by jstarfish »

leetcrew wrote: post #106
jstarfish wrote: post #84

Earlier quoted context omitted.

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the fi…

> How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a…

It does. That's the problem.

Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident.

The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license.

Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts?

(And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#137

Post by waterhouse »

ptd wrote: post #96
stcredzero wrote: post #73

Earlier quoted context omitted.

representatively fair first and foremost, with procedurally fair and utilitarianism at 2nd and 3rd priorities This sounds like a description of Harvard's admission policies with regards to Asians.

Are Asians underrepresented at Harvard? Or if you’re referring to something else, can you elobarate?

As I understand it, Asians are "underrepresented" at Harvard relative to what they would be if Harvard judged candidates based on SAT scores, scholastic achievement, and other seemingly-objective-and-reasonably-sensible metrics. They're "underrepresented" relative to the pool of candidates who are qualified to be at Harvard. But they are "overrepresented" compared to the general population of the United States. Which comparison is the right one?

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#138

Post by burfog »

jstarfish wrote: post #84
shadowbained wrote: post #30

Earlier quoted context omitted.

The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide people into groups in order to perform some sort of balancing? Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loa…

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the fi…

Your higher rate makes a bit of sense. The worst problem is the small sample size. If everybody crashed a minimum of 100 times, then the risk estimates would be much better.

The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#139

Post by malvosenior »

cmiles74 wrote: post #125
malvosenior wrote: post #109

Earlier quoted context omitted.

> If the intent of the perpetrator is the deciding factor, then it is always arguable that perpetrator's intent was not specifically racist; perhaps it was driven by a misunderstanding, etc. Since it was informed by misunderstanding, it isn't racist. Are you saying that this is racism. Most people would define what you outlined as not rascist.

I'm saying that racism cannot be solely an issue of someone's intent, it needs to be evaluated by the outcome. An outcome is a fact that we can all observe. An intent is internal to a person, we can never get a clear view of a person's intent; at best we can make deductions as to a person's intent based on their behavior. Claiming that racism can be evaluated only by the intent is simply moving the goal posts into an…

> Claiming that racism can be evaluated only by the intent is simply moving the goal posts into an area where we can't clearly observe. It's a tactic.

Claiming racism is anything but intent is changing the definition of racism. Which is:

"prejudice, discrimination, or antagonism directed against someone of a different race based on the belief that one's own race is superior."

If you start changing the definition of words to suit a political goal, only the people who already agree with you will listen.

Re: Alexandria Ocasio-Cortez Is Absolutely Right About Racist Algorithms

#140

Post by dgzl »

dleslie wrote: post #97
dgzl wrote: post #79

Earlier quoted context omitted.

> We could benefit from better language around these concepts I believe this is a very under-rated problem today's public is facing. Not just these concepts, but many political terms are misused in today's climate, making for a conversationally ignorant population.

I feel like a writer once discussed the power that exists in limiting and controlling the language used in public discourse... /s

Doublespeak is too real.
Post reply on HN