Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on AI Bias

ben-evans.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: Notes on AI Bias

#61

Post by layoutIfNeeded »

twa927 wrote: post #46
pedrosorio wrote: post #42

Earlier quoted context omitted.

Machine learning was extensively used in image processing before 2013 / deep learning. The main difference is that you’d write code to extract features from the image and then learn a model using those features (as opposed to using the pixel data directly and learning a model from that as in CNNs). As an example, you wouldn’t necessarily write code for “fur texture” but instead would extract histograms of pixel brigh…

Ok, so this depends on what algorithms are used for the feature detection ("edges in an image, and an eye detector, and a texture analyser for fur"). I'm guessing hand-coding an algorithm for detecting edges in an image can be done successfully, but it looks much harder for "an eye detector", so it needs "machine learning" What I meant when asking for a name of an algorithm class are algorithms where the feature extr…

You can call them “handcrafted decision trees” then.

Re: Notes on AI Bias

#62

Post by pixl97 »

Bartweiss wrote: post #59
voidifremoved wrote: post #47

Earlier quoted context omitted.

A while ago, Google photos autogenerated a video for me from my photo library. It was about a minute long, stitched together dozens of photos, called "dog video", and with a horrifying yapping dog soundtrack. Every single photo was of a cat. I have to say I was humbled by the amount of human and computing power that had gone into developing this system over the years, that could achieve such a complicated, impressive…

> be 100% wrong This really is quite impressive. It's rare for humans to do worse than random guessing on tasks, and they almost never do much worse. There's something almost charming about the ability of AI to put real effort into actively avoiding correct answers.

Really it sounds like an error somewhere else, rather than the AI system. More like the ID for cat and dog were switched.

Re: Notes on AI Bias

#63

Post by [deleted] »

FakeComments wrote: post #16
TheCoelacanth wrote: post #11

Earlier quoted context omitted.

I highly doubt that the preference for male candidates was the only problem with the AI. The preference for candidates with ice hockey on their resume almost certainly also would have resulted in a preference for white candidates.

Sure, my point is why does that matter? The correlation with ice hockey could be a career relevant detail because of correlation with discipline and pain tolerance. If it tracks all such signals, I don’t see the problem. That it also has a circumstantial correlation with races isn’t inherently problematic — it could be that different groups of people are differently qualified. There seems to be a position that cultur…

[deleted]

Re: Notes on AI Bias

#64

Post by 6cd6beb »

bduerst wrote: post #60
gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

If you train an AI using data from a system that already has certain biases, then the AI is going to replicate those same systemic biases in it's own predictions. It follows the "garbage in, garbage out" idiom. Curiously though, did you compare the non-hire (full time) rates of interns vs fire rates of non-interns?

>If you train an AI using data from a system that already has certain biases, then the AI is going to replicate those same systemic biases

That's not what happened in the example at all. The example company isn't biased against summer interns, "who stops working after x time" was just a bad question.

The comment you're replying to can boil down to "do you want a monkey's paw solving your problem? If so then AI may be for you"

Or perhaps "stop pretending you're ever going to get ethics or empathy out of a computer"

Re: Notes on AI Bias

#65

Post by Bartweiss »

DuskStar wrote: post #21
fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

One situation I could see leading to this result (Amazon cancelling their resume filtering software with the excuse that it 'skewed male') is that 1. The AI system accurately predicted employee success across both genders AND 2. The AI system predicted that women would do worse than men That's politically embarrassing and something that you can't necessarily 'fix' by improving the system. (see: all the 'will this per…

In this case, it appears to instead be a matter of journalists focusing on totally the wrong aspect of a story for more drama. Buried deep in the original Reuters piece is this offhand mention:

> Gender bias was not the only issue. Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returning results almost at random, Amazon shut down the project, they said.

Apparently the recommendation system really did create gender bias, neither inherited from real differences nor from replicated human biases. (It looks like an issue with mismatched training data and task.) But that initial bias was found and corrected (2015) more than a year before the project was cancelled (2017) for providing "random" results. I think this is the most extreme case of algorithmic bias I've ever seen, but also the least commonly relevant; Amazon appears to have built a model which contained almost no rules except sexism, and scrapped it for not knowing anything worthwhile.

https://www.reuters.com/article/us-amazon-com-jobs-automatio...

Re: Notes on AI Bias

#66

Post by FakeComments »

michaelmior wrote: post #32
FakeComments wrote: post #16

Earlier quoted context omitted.

Sure, my point is why does that matter? The correlation with ice hockey could be a career relevant detail because of correlation with discipline and pain tolerance. If it tracks all such signals, I don’t see the problem. That it also has a circumstantial correlation with races isn’t inherently problematic — it could be that different groups of people are differently qualified. There seems to be a position that cultur…

Race and culture are two very different things. There certainly can be some correlation, but drawing conclusions of suitability for employment based on race is a problem.

I agree — race and culture are very different things.

That’s why it’s only meaningful to draw these conclusions once you control for cultural variation.

No one has shown that they drew conclusions based on race: they drew conclusions based on written words, which correlated with race. The claim is that this demonstrates racial discrimination.

However, an equally valid explanation is culture as a confounding variable correlated with both race and hiring, and which is the actual causative factor.

My point is that no one has shown this is hiring based on race, because the only things being judged are cultural artifacts. It’s merely presumed that anything correlated with race is inherently racist, when that’s not true.

No one has even remotely tried to address that, including yourself, preferring to merely assume it’s racism or special pleading like “well, there is some correlation — but without doing the math, I know that can’t be it!”

(All these same point apply to sexism.)

Re: Notes on AI Bias

#67

Post by Bartweiss »

pixl97 wrote: post #62
Bartweiss wrote: post #59

Earlier quoted context omitted.

> be 100% wrong This really is quite impressive. It's rare for humans to do worse than random guessing on tasks, and they almost never do much worse. There's something almost charming about the ability of AI to put real effort into actively avoiding correct answers.

Really it sounds like an error somewhere else, rather than the AI system. More like the ID for cat and dog were switched.

If this is the Google Photos folder system, I suspect the problem was that the IDs and the bucketing were de-linked.

Photos creates folders for you based on identified themes, and then adds new photos to them as they're taken. I haven't checked, but I'm guessing it doesn't relabel existing buckets to avoid causing confusion. And I'm not sure whether bucketing is done by assessing theme or similarity to other photos in a folder. If it's the latter, the system could have hit the confidence threshold to make a Dog folder out of a few images, then ceaselessly dumped similar-looking photos (i.e. cats) into that bucket.

Re: Notes on AI Bias

#68

Post by jakelazaroff »

gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

> There is nothing "mechanistic" about this. It depends on how you select sample resumes and how you split them between "good" and "bad" labels.

Isn't that what the article is trying to say, though? That your model can only be as accurate as your data set… and that even then, you have to be very careful to make sure it's not inferring patterns from entirely unrelated information?

Re: Notes on AI Bias

#69

Post by bduerst »

6cd6beb wrote: post #64
bduerst wrote: post #60

Earlier quoted context omitted.

If you train an AI using data from a system that already has certain biases, then the AI is going to replicate those same systemic biases in it's own predictions. It follows the "garbage in, garbage out" idiom. Curiously though, did you compare the non-hire (full time) rates of interns vs fire rates of non-interns?

>If you train an AI using data from a system that already has certain biases, then the AI is going to replicate those same systemic biases That's not what happened in the example at all. The example company isn't biased against summer interns, "who stops working after x time" was just a bad question. The comment you're replying to can boil down to "do you want a monkey's paw solving your problem? If so then AI may be…

I was referring to the Amazon resume model. The intern hires model was labeling, as GP said.

Re: Notes on AI Bias

#70

Post by lonnyk »

gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

I get the point, but why didn't you just exclude intern resumes from the training data? Do you still suspect a skewed result?
Post reply on HN