Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on AI Bias

ben-evans.com

Post reply on HN
Print view View original

41–50 of 126 posts

Re: Notes on AI Bias

#41

Post by MiroF »

munificent wrote: post #30
eanzenberg wrote: post #9

>>Now, suppose that 75% of the bad turbines use a Siemens sensor and only 12% of the good turbines use one (and suppose this has no connection to the failure). The system will build a model to spot turbines with Siemens sensors. Oops. Given a statistically large enough sample, 2 outcomes: 1) The Siemens sensor actually is at fault. 2) The Siemens sensor is a part of a larger system, which is different in non-Siemens…

I think you assume here that the historical effects that led to Siemens sensors correlating with failure will continue to be true in the future. And I think that is the key fallacy that makes AI bias a problem. We aren't just looking for patterns. We are looking for patterns so that we can take action and affect the future. If the patterns, which are real enough in the historical data, don't correctly predict the imp…

> I think you assume here that the historical effects that led to Siemens sensors correlating with failure will continue to be true in the future.

Yes, AI systems presume induction to be true. But so does... uh, science and most other things we do?

Re: Notes on AI Bias

#42

Post by pedrosorio »

twa927 wrote: post #36
layoutIfNeeded wrote: post #26

Earlier quoted context omitted.

I would call it “classical” machine learning.

Hmm, I think there's no "machine learning" here. There's a human hard-coding some thought process, using mostly some simple statistics/thresholds to e.g. define what a "fur texture" looks like.

Machine learning was extensively used in image processing before 2013 / deep learning.

The main difference is that you’d write code to extract features from the image and then learn a model using those features (as opposed to using the pixel data directly and learning a model from that as in CNNs).

As an example, you wouldn’t necessarily write code for “fur texture” but instead would extract histograms of pixel brightness gradients and feed those (along with other things) to a machine learning algorithm. In this example, fur texture would generate a different histogram (to be used as a feature) than skin texture.

https://en.m.wikipedia.org/wiki/Histogram_of_oriented_gradie...

Re: Notes on AI Bias

#43

Post by gizmo686 »

DuskStar wrote: post #21
fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

One situation I could see leading to this result (Amazon cancelling their resume filtering software with the excuse that it 'skewed male') is that 1. The AI system accurately predicted employee success across both genders AND 2. The AI system predicted that women would do worse than men That's politically embarrassing and something that you can't necessarily 'fix' by improving the system. (see: all the 'will this per…

Your example is quite possible, particularly at an organazation that would be embarrased by such a result.

Assume that the ability curve of male applicants and female applicants are identical; that the majority of applicants are male; and that Amazon wants to hire more females then would be expected given the portion of applicants that are female.

A natural way of accomplishing this goal is to give extra points to female applicants [0].

Due to selection bias, the ability curve of women within the population of Amazon engineers would skew lower then men within the population of Amazon engineers.

This is a special case of a more general phenomona. If you have signal S that is positivly correlated with a desired trait in the general population, and over select for S, you will find that S is negativly correlated within your population.

[0]. All proposals I have seen amount to either a good approximation of this or changing the applicant pool. And, by assumption, the latter is excluded.

Re: Notes on AI Bias

#44

Post by DuskStar »

throwawaymath wrote: post #19
eanzenberg wrote: post #9

>>Now, suppose that 75% of the bad turbines use a Siemens sensor and only 12% of the good turbines use one (and suppose this has no connection to the failure). The system will build a model to spot turbines with Siemens sensors. Oops. Given a statistically large enough sample, 2 outcomes: 1) The Siemens sensor actually is at fault. 2) The Siemens sensor is a part of a larger system, which is different in non-Siemens…

No, that's incorrect. Note the part of your quote which says, "and suppose this has no connection to the failure." The point is the Siemens sensor is a superfluous correlation with turbine failure, because the underlying dataset is biased towards Siemens sensors. The scenario suggested by the author is one in which your turbine failure dataset does not match reality. No amount of sample enlargement will correct sampl…

Just to point out:

1. Not a spurious correlation - Siemens sensors are in fact associated with increased failure rates in the dataset and if you continue to sample data with the same methodology this correlation will continue. You need to fix your data collection methodology, but it's not a spurious correlation.

2. See #1.

3. See #1.

4. The original problem statement said that a low percentage of unfailed turbines used Siemens sensors, and a high percentage of failed turbines used Siemens sensors. So 'you use mostly Siemens sensors' would imply that most of your turbines have failed, which seems a little unlikely to me.

Re: Notes on AI Bias

#45

Post by AJRF »

twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

Maybe image segmentation? In my AI class it was referred to as image segmentation and edge detection (interchangeably)

https://en.wikipedia.org/wiki/Image_segmentation

Re: Notes on AI Bias

#46

Post by twa927 »

pedrosorio wrote: post #42
twa927 wrote: post #36

Earlier quoted context omitted.

Hmm, I think there's no "machine learning" here. There's a human hard-coding some thought process, using mostly some simple statistics/thresholds to e.g. define what a "fur texture" looks like.

Machine learning was extensively used in image processing before 2013 / deep learning. The main difference is that you’d write code to extract features from the image and then learn a model using those features (as opposed to using the pixel data directly and learning a model from that as in CNNs). As an example, you wouldn’t necessarily write code for “fur texture” but instead would extract histograms of pixel brigh…

Ok, so this depends on what algorithms are used for the feature detection ("edges in an image, and an eye detector, and a texture analyser for fur"). I'm guessing hand-coding an algorithm for detecting edges in an image can be done successfully, but it looks much harder for "an eye detector", so it needs "machine learning"

What I meant when asking for a name of an algorithm class are algorithms where the feature extraction is done using hand-coded algorithms.

Re: Notes on AI Bias

#47

Post by voidifremoved »

twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

A while ago, Google photos autogenerated a video for me from my photo library. It was about a minute long, stitched together dozens of photos, called "dog video", and with a horrifying yapping dog soundtrack.

Every single photo was of a cat.

I have to say I was humbled by the amount of human and computing power that had gone into developing this system over the years, that could achieve such a complicated, impressive technical feat, without requiring any effort or money on my part, and yet also be 100% wrong.

Re: Notes on AI Bias

#48

Post by gizmo686 »

turtlecloud wrote: post #31
fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

Just remove the gender/sex as variables for the AI and maybe name too. Preprocess the resumes to remove them. Now you remove the majority of gender bias for the AI.

AI is really good at infering information. If gender is a real signal, it would be very difficult to filter the input such that it is not making a determination by what could be reffered to inferred gender.

Re: Notes on AI Bias

#49

Post by throwawaymath »

DuskStar wrote: post #39
throwawaymath wrote: post #33

Earlier quoted context omitted.

You really should. If the sample is "all the gas turbines you own" and you disproportionately use Siemens sensors, your turbine failure forecast will (with high likelihood) reduce to a Siemens sensor forecast. This is easily plausible even if your sample's correlation between Siemens sensors and gas turbines is completely superfluous.

You can't have a sampling bias when 'sampling' the entire population, because the definition of 'sampling bias' includes 'some members are not included in the sample'.

Precisely, yes. I'm talking about a sample including all representative gas turbine failures, across all sensor vendors.

Re: Notes on AI Bias

#50

Post by lotu »

DuskStar wrote: post #21
fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

One situation I could see leading to this result (Amazon cancelling their resume filtering software with the excuse that it 'skewed male') is that 1. The AI system accurately predicted employee success across both genders AND 2. The AI system predicted that women would do worse than men That's politically embarrassing and something that you can't necessarily 'fix' by improving the system. (see: all the 'will this per…

This is feels like an elephant in the room when it comes to AI bias. We develop an AI that accurately predicts outcomes and discover it is biased, then instead of asking if maybe this means our current system is deeply biased and needs to be changed, we say, "don't use the AI; keep using the people who might or might not be biased but we don't know because we can't measure it in the way an AI can be measured."

If it isn't acceptable to use an AI to create biased outcomes how is it acceptable to use people to create the the same outcomes. AI decision making can be examined and tuned in ways that people cannot.

Post reply on HN