Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Awful-AI: A curated list to track current scary usages of AI

github.com

Post reply on HN
Print view View original

31–40 of 48 posts

Re: Awful-AI: A curated list to track current scary usages of AI

#31

Post by archi42 »

curiousllama wrote: post #23
centimeter wrote: post #14

A huge portion of these seem to be "I don't like the fact that this heuristic is strongly predictive", not any sort of complaint about the implementation or use of the prediction engine.

Yea, I mean there's a lot of strongly-predictive heuristics that are unethical to act on, though (e.g., "if applicant is pregnant, then she will take maternity leave, and therefore be less productive employee year 1, so don't hire" Why shouldn't automated decision-making processes be held to the same ethical standards as manual ones?

That's an important point, no matter if the algorithm is some trained neural net or a ruleset written by hand.

I'm often under the impression that moral standards are easier to ignore if "the computer says that X". E.g. we have to accept the fact that woman is much more likely to take maternety leave, and essentially an AI is okay to be aware of that, too. It even has to, else it finds another commom factor (eg womans chess club) - but it should be able to wilfully ignore this finding because using it against an applicant is immoral.

Re: Awful-AI: A curated list to track current scary usages of AI

#32

Post by gitgud »

pineman wrote: post #4

Is there a list of Nice-AI? Or generally unambiguously awesome tech?

If technology is a tool to make something easier, then I would say there's no such thing as "unambiguously awesome tech", as all tech can be used for good or evil...

Re: Awful-AI: A curated list to track current scary usages of AI

#34

Post by elisbce »

curiousllama wrote: post #18
convery wrote: post #2

Ehh.. the discrimination part sounds a lot like: "we had an AI look at statistics and it reported the patterns we choose to ignore".

Ah, yes, but that's not really the point, is it? COMPAS points out an uncomfortable trend ("violent crime is more prevalent in black communities...") but applies it in a way that is bad ("...and therefore black people should get longer criminal sentences for the same crime"). We both agree, I'm sure, that socioeconomics and race shouldn't meaningfully impact sentence length. We may also agree that it often does. What…

You are deliberately twisting it. The crime stats says "violent crime is more prevalent in black communities...", but no AI/ML will produce the conclusion "...and therefore black people should get longer criminal sentences for the same crime". They are completely irrelevant. The AI model will simply conclude that race would be a factor when you are looking at crime rate, or posterior probability when a violent crime happens. And that is exactly accurate. What's wrong with that? That's a fact. Math doesn't lie nor does it have any agenda. There is nothing the AI is making up or adding up to. It is pure statistics and probability. It is the human beings that can't take the truth and the consequences.

Re: Awful-AI: A curated list to track current scary usages of AI

#35

Post by elisbce »

lxe wrote: post #24
convery wrote: post #2

Ehh.. the discrimination part sounds a lot like: "we had an AI look at statistics and it reported the patterns we choose to ignore".

The issue is using the correlation to make predictions. You're just perpetuating the trend instead of trying to change it.

It has nothing to do with AI/ML. If you see someone repeatedly stealing from your yard, you can't predict he would do it again with high probability, because it would perpetuate the trend instead of trying to change it? Why would the responsibility of change lie on the observers instead of the perpetrators? If he cares about his image, he can stop stealing and with time, people would conclude so.

Re: Awful-AI: A curated list to track current scary usages of AI

#36

Post by elisbce »

SpicyLemonZest wrote: post #21
centimeter wrote: post #14

A huge portion of these seem to be "I don't like the fact that this heuristic is strongly predictive", not any sort of complaint about the implementation or use of the prediction engine.

Fairness and predictivity are different concerns. Many of my interests, hobbies, and behaviors are strongly predictive of poor public speaking skills, but it would still be very unfair if my company installed an AI saying I can't give conference speeches because I'll probably do poorly.

1) Why is unfair? If there is one speech, but 3 candidates. And the AI predicted based on past performances, why is it unfair? (It is a bad decision for you, or the AI could be making a poor decision based on data/algorithm limitations, but it is not unfair, the AI doesn't favor anyone outside of the data it's given). 2) AI doesn't make the decision that you can't give speeches. AI gives truthful predictions assuming the data is truthful. But it is your boss who uses the results to make that decision. The ethics part is on your boss using what data/algo to make such a prediction to determine who should go. Why would you blame the AI instead of the boss?

Re: Awful-AI: A curated list to track current scary usages of AI

#38

Post by curiousllama »

elisbce wrote: post #34
curiousllama wrote: post #18

Earlier quoted context omitted.

Ah, yes, but that's not really the point, is it? COMPAS points out an uncomfortable trend ("violent crime is more prevalent in black communities...") but applies it in a way that is bad ("...and therefore black people should get longer criminal sentences for the same crime"). We both agree, I'm sure, that socioeconomics and race shouldn't meaningfully impact sentence length. We may also agree that it often does. What…

You are deliberately twisting it. The crime stats says "violent crime is more prevalent in black communities...", but no AI/ML will produce the conclusion "...and therefore black people should get longer criminal sentences for the same crime". They are completely irrelevant. The AI model will simply conclude that race would be a factor when you are looking at crime rate, or posterior probability when a violent crime…

You're right, of course: "human beings cant take the truth and consequences." "That is exactly accurate [...] That's a fact" - both true. I think the issue is less truth and more action, as in "I believe it to be immoral to act upon that fact in manner X."

I can come up with a dozen logical statements that are equally correct and immoral to act upon in certain ways. If an ML algorithm had come up with them, they would be no less correct and no less immoral. For example, "If a woman applicant is pregnant, she will take maternity leave and be less productive year 1. Therefore, we should not hire her." COMPAS drew a number of objective conclusions and ties them to specific recommendations in its intended use: (1) if you're poor, you're more likely to recidivate. Therefore, you should get a longer sentence. (2) if you live in a black community, you're more likely to recidivate. Therefore, you should get a longer sentence (3) if you're related to a gang member (not just one yourself - even if your estranged brother joined a gang a decade ago), you're more likely to recidivate. Therefore, you should get a longer sentence.

Notice: the "you should get a longer sentence" bit is actually part of the tool. That was the intervention it was designed to suggest. There's a number of critiques of that connection (mathematical, legal, moral), but it's not mine, nor is it the "AI's"; it's Northpointe's. And ultimately that's the scandal & intrigue: the math is right logically, but even so, the tool's recommendation is wrong ethically.

Post reply on HN