Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on AI Bias

ben-evans.com

Post reply on HN
Print view View original

31–40 of 126 posts

Re: Notes on AI Bias

#31

Post by turtlecloud »

fvdessen wrote: post #2

> Since Amazon’s current employee base skews male, the examples of ‘successful hires’ also, mechanistically, skewed male and so, therefore, did this system’s selection of resumés. Amazon spotted this and the system was never put into production. Couldn't they have retrained the system with a 50/50 mix of males / females resumes ? Or restrict the use of the algorithm to sort male resumes ? Or maybe resumes don't actua…

Just remove the gender/sex as variables for the AI and maybe name too. Preprocess the resumes to remove them. Now you remove the majority of gender bias for the AI.

Re: Notes on AI Bias

#32

Post by michaelmior »

FakeComments wrote: post #16
TheCoelacanth wrote: post #11

Earlier quoted context omitted.

I highly doubt that the preference for male candidates was the only problem with the AI. The preference for candidates with ice hockey on their resume almost certainly also would have resulted in a preference for white candidates.

Sure, my point is why does that matter? The correlation with ice hockey could be a career relevant detail because of correlation with discipline and pain tolerance. If it tracks all such signals, I don’t see the problem. That it also has a circumstantial correlation with races isn’t inherently problematic — it could be that different groups of people are differently qualified. There seems to be a position that cultur…

Race and culture are two very different things. There certainly can be some correlation, but drawing conclusions of suitability for employment based on race is a problem.

Re: Notes on AI Bias

#33

Post by throwawaymath »

DuskStar wrote: post #15
TheCoelacanth wrote: post #13

Earlier quoted context omitted.

Only if your test data is free of sample bias. Given how incredibly hard it is to avoid sample bias, you can't take it for granted that your training data doesn't have any sample bias.

If the sample is "all the gas turbines I own", I don't particularly CARE about the bias...

You really should. If the sample is "all the gas turbines you own" and you disproportionately use Siemens sensors, your turbine failure forecast will (with high likelihood) reduce to a Siemens sensor forecast. This is easily plausible even if your sample's correlation between Siemens sensors and gas turbines is completely superfluous.

Re: Notes on AI Bias

#34

Post by twa927 »

piker wrote: post #29
twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

Expert systems? https://en.wikipedia.org/wiki/Expert_system

I think expert systems consist of a "rule engine" where rules can be added dynamically?

Re: Notes on AI Bias

#35

Post by throwawaymath »

DuskStar wrote: post #28
TheCoelacanth wrote: post #22

Earlier quoted context omitted.

If the training data is all gas turbines that you own, why do you care about having the ML model at all? You already have complete knowledge of the state of all your gas turbines. There's no point to having an ML model unless you are applying it to something outside of the training data. If you plan on applying the model to different turbines, then there is potential for sample bias in which turbines you selected. If…

Well, I might care about predicting the next turbine to fail. If Siemens sensors are truly unrelated to the issues, that'll average out eventually - but I'd be highly skeptical of someone asserting that it's completely unrelated to the failures and not just covarying with something we're not using as a model input. Why would I care about the fact that only 10% of turbines globally have Siemens sensors? I don't know t…

The superfluous correlation between Siemens sensors and turbine failures will not average out eventually if you have a sampling bias in your dataset.

Re: Notes on AI Bias

#36

Post by twa927 »

layoutIfNeeded wrote: post #26
twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

I would call it “classical” machine learning.

Hmm, I think there's no "machine learning" here. There's a human hard-coding some thought process, using mostly some simple statistics/thresholds to e.g. define what a "fur texture" looks like.

Re: Notes on AI Bias

#37

Post by frankbreetz »

twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

First order logic rule-based system

Re: Notes on AI Bias

#38

Post by DuskStar »

throwawaymath wrote: post #35
DuskStar wrote: post #28

Earlier quoted context omitted.

Well, I might care about predicting the next turbine to fail. If Siemens sensors are truly unrelated to the issues, that'll average out eventually - but I'd be highly skeptical of someone asserting that it's completely unrelated to the failures and not just covarying with something we're not using as a model input. Why would I care about the fact that only 10% of turbines globally have Siemens sensors? I don't know t…

The superfluous correlation between Siemens sensors and turbine failures will not average out eventually if you have a sampling bias in your dataset.

You keep saying that I have a sampling bias, but there really isn't any evidence for that. I'm sampling 100% of the population. You can't have a sampling bias when sampling 100% of the population.

It could be a spurious correlation, sure - but that'll go away as the amount of data increases.

Re: Notes on AI Bias

#39

Post by DuskStar »

throwawaymath wrote: post #33
DuskStar wrote: post #15

Earlier quoted context omitted.

If the sample is "all the gas turbines I own", I don't particularly CARE about the bias...

You really should. If the sample is "all the gas turbines you own" and you disproportionately use Siemens sensors, your turbine failure forecast will (with high likelihood) reduce to a Siemens sensor forecast. This is easily plausible even if your sample's correlation between Siemens sensors and gas turbines is completely superfluous.

You can't have a sampling bias when 'sampling' the entire population, because the definition of 'sampling bias' includes 'some members are not included in the sample'.
Post reply on HN