What if we trained a bunch of robots with AIs embedded and they turned out to be racist misogynists? There has got to be a science fiction story about that...
You mean like Tay?
Amazon scraps secret AI recruiting tool that showed bias against women
371–380 of 433 posts
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#372The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…
The term "AI" is over-hyped. What we have now is advanced pattern recognition, not intelligence. Pattern recognition will learn any biases in your training data. An intelligent enough* being does much more than pattern recognition -- intelligent beings have concepts of ethics, social responsibility, value systems, dreams, ideals, and is able to know what to look for and what to ignore in the process of learning. A du…
We are pattern recognition machines. If you consider pattern matching unintelligent, then machines are more intelligent that we are since they rely more on logic than pattern matching.
For the black man = gorilla problem, an untaught human, a small child for instance, can easily make the same mistake. Especially if he has seen few black people. And well educated adults can also make the mistake initially, even if they hate to admit it.
However, in the last case, a second pattern recognition happen, one that matches the result of the image classifier with social rules. And it turns out that mixing black men and gorillas is a clear anti-pattern and anything that isn't certain is incorrect.
Unlike us, computer image classifiers typically aren't taught social rules, so like a small child, they will tell things without filter. It will probably change in the future for public facing AIs.
Not stereotyping is not a mark of intelligence, it is a mark of a certain type of education. And I don't see why it couldn't be done with the usual machine learning techniques.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#373The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…
The article didn't specify how they labeled resumes for training. You're assuming that it was based on whether or not the candidate was hire. Nobody with an iota of experience in machine learning would do something like that. (For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.) A far more reasonable way would be to take resumes of people who were hired and train the mod…
> You're assuming that it was based on whether or not the candidate was hire. Nobody with an iota of experience in machine learning would do something like that. (For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.)
It's a fine strategy if all you're trying to do is cost-cut and replace the people that currently make these decisions (without changing the decisions).
I agree that most people with ML experience would want to do better, and could think of ways to do so with the right data, but if all the data that's available is "resume + hire/no-hire", then this might be the best they could do (or at least the limit of their assignment).
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#374I'm honestly pretty shocked that Amazon went ahead with this which is such an obvious legal liability in an extremely touchy area of business practice. Everything about this should have been a red flag that meant it never should have gotten off the ground, let alone been in place for years.
You don't think they should try new things, even though they might not be successful?
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#375Earlier quoted context omitted.
Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process? Maybe - but maybe not. Imagine a company with 2 men in HR, 2 women in HR, 40 men in engineering, and 10 women in engineering. That's with gender-blind hiring, reflecting only the 4:1 ratio of male to female CS graduates. If you picked a random male hire, there's a 40/42=95% chance they're an engineer whereas if you p…
Why would one train on "all hires CVs"? It'd be "engineering CVs", moreover it'd be "engineering applicants CVs", not "engineering hires CVs".
Because we are guessing at the problem, we don't have complete data.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#376The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…
The article didn't specify how they labeled resumes for training. You're assuming that it was based on whether or not the candidate was hire. Nobody with an iota of experience in machine learning would do something like that. (For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.) A far more reasonable way would be to take resumes of people who were hired and train the mod…
> For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.
Many companies are fine with false negatives in their hiring process. Better to pass on a good candidate than hire a bad one.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#377Hold on here. This article seems to have buried a pretty important piece of information wayyy down in the middle of the text. > Gender bias was not the only issue. Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returning results almost at random, Amazon shut down the project, they said.…
The media is no longer reporting things. You can't make money with reporting. The media is actively creating narratives, and one of the narratives that people are fed nowadays is that women are victims. Men and women are pitted against each other. Due to the way the media has evolved people consume their own biases and most often just read the headlines.
The media has always been about narratives.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#378The thought that you can somehow suss out with any consistent performance who will be good or not at their job based solely on a training set consisting of resumes is so banal and inept that its laughable. Team building has so much more to do with assembling the right kinds of people with the right talent and personalities that no present day machine learning could ever hope to come close in effectiveness to a thoughtfully applied interviewing and hiring process.
It is clear whatever executives in charge of this project haven't the first clue of whatever it is they are doing. It lacks not only any technical deliberateness but also fails the common sense test. I really wouldn't want to be working for such people.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#379Earlier quoted context omitted.
Agreed. There is no story here.
The story is, some ML researchers did their job properly and detected ethical issues before they became a problem. That's more rare than you'd think.
Yes. Software engineers taking ethics seriously and not letting technical enthusiasm blind them is news, not normality.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#380Earlier quoted context omitted.
The term "AI" is over-hyped. What we have now is advanced pattern recognition, not intelligence. Pattern recognition will learn any biases in your training data. An intelligent enough* being does much more than pattern recognition -- intelligent beings have concepts of ethics, social responsibility, value systems, dreams, ideals, and is able to know what to look for and what to ignore in the process of learning. A du…
We are pattern recognition machines. If you consider pattern matching unintelligent, then machines are more intelligent that we are since they rely more on logic than pattern matching. For the black man = gorilla problem, an untaught human, a small child for instance, can easily make the same mistake. Especially if he has seen few black people. And well educated adults can also make the mistake initially, even if the…
> social rules
I claim it isn't just social rules -- part of that is empathy, which is a manifestation of intelligence that I think is beyond pattern matching.
If a white person were mislabeled as a cat, it would be a cute funny mistake. Labeling people as dogs, not so much. Gorillas, even worse. Despite that gorillas are more intelligent and empathetic than cats. Oh, and bodybuilder white celebrity boxing champion as a gorilla, may actually be okay. The same guy as a dog, no. It makes no sense to a logic-based algorithm. But humans "get it".
A human gets it because they could imagine the mistake happening against them, with absolutely zero prior training data. You don't need to have seen 500 examples of people being called gorillas, cats, dogs, turtles and whatever else.
If you want to say that a hundred pattern recognition algorithms working together in a delicate way might manifest intelligence, I think that is possible. But the point is one task-specific lowly pattern recognition algorithm, which is today's state of the art, is pretty stupid.