I actually think this is where ML really shines. You can pick things apart. Sure, you might need carefully designed experiments, but you can subtract "female" from the resume and look for other data that cause some trained machine to activate, like patterns of word choice, etc. This is akin to the Go players learning from Alpha Go. It's actually a richly rewarding investigation for those of us who have done it. To discover a whole class of failure modes, that's success! And, unlike courts of law, the the process is much more efficient, because you don't have to contend with a defendant appealing to matters of intent or the emotions of a jury.
Notes on AI Bias
101–110 of 126 posts
Re: Notes on AI Bias
#102This quote stands out to me: "just as a dog is much better at finding drugs than people, but you wouldn’t convict someone on a dog’s evidence. And dogs are much more intelligent than any machine learning." Because in my head I followed it with the sentence "but we're all confident that we will have dogs driving our cars in about 5 years." Food for thought for sure.
So dogs are better than humans at detecting drugs because they have a better sense of smell than can penetrate packaging. What does that have to do with technology being better/worse than humans at driving, exactly?
They didn't say dogs were better than technology at solving problems, in any sort of general sense.
Re: Notes on AI Bias
#103Earlier quoted context omitted.
>Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? In my opinion, yes, if it leads most readers to misjudge some fundamental properties of the problem as a whole. Again, I'm not saying this article is guilty, but most are.
> In my opinion, yes, if it leads most readers to misjudge some fundamental properties of the problem as a whole. Which problem? The general statement of this problem is "models, trained on [somehow] misrepresentative data [or even technically representative data] can draw unintended conclusions that lead to harm". Specifically in this case, the harm was "the model was basically just trained to ignore all women appli…
[deleted]
Re: Notes on AI Bias
#104Earlier quoted context omitted.
You really should. If the sample is "all the gas turbines you own" and you disproportionately use Siemens sensors, your turbine failure forecast will (with high likelihood) reduce to a Siemens sensor forecast. This is easily plausible even if your sample's correlation between Siemens sensors and gas turbines is completely superfluous.
You can't have a sampling bias when 'sampling' the entire population, because the definition of 'sampling bias' includes 'some members are not included in the sample'.
You can't make predictions when sampling the entire population.
Re: Notes on AI Bias
#105>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…
>Framing this problem as "bias" Except that's exactly what it is. Much as your model was biased against interns. > and especially hyper-focusing everyone's attention on diversity aspect of it is extremely irresponsible. Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? Just because the same kind of methodological flaw can cause other harms its irresponsible to use a motiv…
> Except that's exactly what it is.
Using the term 'bias' has certain political motivations behind it. It's not about the term being technically untrue as it is about the term being non-neutral. For instance, here are some definitions of 'bias' I just grabbed from American Heritage:
"A preference or an inclination, especially one that inhibits impartial judgment."
"An unfair act or policy stemming from prejudice."
"A statistical sampling or testing error caused by systematically favoring some outcomes over others."
The ML model does not have a preference, inclination, or prejudice relating to interns, except insofar as we anthropomorphize it to have them. What does using a word suggesting that add?
A more neutral account of what's going on is along the lines: It's easy to accidentally train ML models so that they will make systematic errors. (Among those errors is the possibility for it to exhibit behavior resembling prejudice.)
Re: Notes on AI Bias
#106Earlier quoted context omitted.
>Framing this problem as "bias" Except that's exactly what it is. Much as your model was biased against interns. > and especially hyper-focusing everyone's attention on diversity aspect of it is extremely irresponsible. Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? Just because the same kind of methodological flaw can cause other harms its irresponsible to use a motiv…
> Except that's exactly what it is. Using the term 'bias' has certain political motivations behind it. It's not about the term being technically untrue as it is about the term being non-neutral. For instance, here are some definitions of 'bias' I just grabbed from American Heritage: "A preference or an inclination, especially one that inhibits impartial judgment." "An unfair act or policy stemming from prejudice." "A…
Fine: it's easy to accidentally train ML models so that they will make systematic errors. Often these errors stem from systematic biases in our society, model creators should therefore be aware of the potential biases[1] that their models could reflect, and how to prevent them.
[1]: With the political motivation.
Re: Notes on AI Bias
#107Earlier quoted context omitted.
You're right, I didn't try to address it. I also didn't make any assumptions of racism in my comment. However, generally speaking, I don't think one culture is better than another either and discrimination based on culture is also a problem. (I understand that this is an opinion and many would disagree with me on that point.)
It’s an absolute nonsense opinion, because there are obviously cultural beliefs that have an impact on work performance, particularly over a lifetime — eg, beliefs in honesty, hard work, and obtaining education. Different cultures place different values on these things, and it leads to different outcomes — even just looking at the impact of cultural drift over time, among a single people. I also find it absolutely ab…
If you keep doing ideological flamewars on HN we will ban you. Also, if you keep doing personal attacks on HN we will ban you.
Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here from now on.
Re: Notes on AI Bias
#108Earlier quoted context omitted.
> Except that's exactly what it is. Using the term 'bias' has certain political motivations behind it. It's not about the term being technically untrue as it is about the term being non-neutral. For instance, here are some definitions of 'bias' I just grabbed from American Heritage: "A preference or an inclination, especially one that inhibits impartial judgment." "An unfair act or policy stemming from prejudice." "A…
Fine: it's easy to accidentally train ML models so that they will make systematic errors. Often these errors stem from systematic biases in our society, model creators should therefore be aware of the potential biases[1] that their models could reflect, and how to prevent them. [1]: With the political motivation.
> Often these errors stem from systematic biases in our society ...
Depending on the what the appropriate quantification of 'often' is, that might make sense. Do we have enough reason to believe it would take on a high enough value to merit the usage of a term that refers only to it?
The other problem with what you're describing is that all we actually know is that the model is reflecting the current state of things. Your statement attributes particular causes to the current state of things, and implies a certain valuation of the current state of things (which I don't personally disagree with, necessarily—but I don't think my personal views should be reflected in scientific/engineering jargon).
So given the uncertain value of 'often,' and the unsettled nature of the causes behind various aspects of the 'current state of things,' it seems to be solidly jumping the gun to frame the entire general problem with a term that refers to this partial and fraught aspect of it.
Re: Notes on AI Bias
#109Earlier quoted context omitted.
Fine: it's easy to accidentally train ML models so that they will make systematic errors. Often these errors stem from systematic biases in our society, model creators should therefore be aware of the potential biases[1] that their models could reflect, and how to prevent them. [1]: With the political motivation.
> Often these errors stem from systematic biases in our society ... Depending on the what the appropriate quantification of 'often' is, that might make sense. Do we have enough reason to believe it would take on a high enough value to merit the usage of a term that refers only to it? The other problem with what you're describing is that all we actually know is that the model is reflecting the current state of things.…
>Your statement attributes particular causes to the current state of things
I didn't, nor should it matter how we got to where we are for a builder of a thing.
> and implies a certain valuation of the current state of things
This may have happened, but I'd disagree: recognizing that there exists inequality doesn't cast value judgement on that inequality. I simply stated that they're there. Perhaps saying "how to prevent them" is casting value judgement, so I might walk that back, model creators should be aware of the biases and aware of tools and strategies to account for them, if so desired.
Personally I think you're a bad person if, armed with the tools to detect and correct, you decide its okay to build something that has a systemic error that wrongly disfavors some group. But perhaps that's just me.
Re: Notes on AI Bias
#110Earlier quoted context omitted.
> Often these errors stem from systematic biases in our society ... Depending on the what the appropriate quantification of 'often' is, that might make sense. Do we have enough reason to believe it would take on a high enough value to merit the usage of a term that refers only to it? The other problem with what you're describing is that all we actually know is that the model is reflecting the current state of things.…
>Your statement attributes particular causes to the current state of things I didn't, nor should it matter how we got to where we are for a builder of a thing. > and implies a certain valuation of the current state of things This may have happened, but I'd disagree: recognizing that there exists inequality doesn't cast value judgement on that inequality. I simply stated that they're there. Perhaps saying "how to prev…
> ... recognizing that there exists inequality doesn't cast value judgement on that inequality.
You just asserted your attribution of cause right there: inequality. There are multiple possible causes for differing demographic representations in various roles. This is not a settled issue, even though people on both sides promote competing ideologies to the effect that it is.
(And again, I have intentionally left my own views on the subject out of this, even though I suspect they align with yours (insofar as cause attribution goes): I'm just pointing out the fact that this isn't something society agrees on, nor is it something the scientific data resolves unambiguously.)
> Personally I think you're a bad person if, armed with the tools to detect and correct, you decide its okay to build something that has a systemic error that wrongly disfavors some group.
Agreed, hinging on that point about cause attribution.