This seems contradictory:
> AI can't predict social outcomes
> In most cases, manual scoring rules are just as accurate
So manual scoring rules don't work either for predicting social outcomes? There is some magic sauce that humans use for prediction that we haven't cracked yet? Nothing can predict social outcome?
AI is perfectly capable of predicting social outcomes, and only in very few cases are manual scoring rules as accurate as black box AI. The ethical concern is not about accuracy, but about our sensibilities when it comes to protected classes. The author cherry picked examples where simpler approaches also worked, but says nothing of practical feasability or increase in variance. Try actually doing face recognition or spam detection with manual rules.
Face recognition being way more accurate is just as much an ethical concern as a gun that is way more accurate. It all depends on who you point it at. Accurate face recognition at the border helps save lives as much as equipping the police with more accurate hand guns.
The talk of AGI is misguided. Everybody can see that the economy will be increasingly automated with narrow AI. Just because "big data" was a hype word, does not mean companies haven't been monitizing their big data (and were thus right to collect it).
We can predict probabilities about the future. The author is attacking these systems for not being 100% sure. Predictive policing is automated resource management. Militaries have been doing this for decades. It has its drawbacks, but also benefits (wiser usage of tax money, protecting low-income neighborhoods from falling in the hands of gangs).
The author also claims that algorithms automatically turn away people at the border for posting or liking or being connected to terrorist propaganda. But these systems just give a score and a human border guard makes the (more informed) decision.
A system not being 100% accurate is not an ethical concern, as long as we not treat those systems as 100% accurate and give proper recourse.
Just a spelling check can and does weed out poor candidates. Why does HR want to automate? Because they get 1000+ resumes for a single position. The manual glance they give them pale in comparison to what an automated system can do.
What is more likely? That these HR systems show promise? Or that the VC market has completely lost it (despite working with software and automation for decades, and have AI experts on staff) and is pumping billions into tealeaf reading, because now its called "AI"?
If you cheat the system by adding "Cambridge" or "Oxford" in white letters to your CV, is that ethical? Why not add it to your education section in black letters? Would you hire a good potential candidate, if you knew they acted like 90s search engine spammers? Maybe a candidate from Oxford or Cambridge really deserves to be on the top of the pile, or is it now unethical to look at education when hiring?
This presentation likes to mix ethics with technical success. Just say that a HR system is unethical, without calling it bogus with zero proof other than "some AI experts agree that this is impossible".
Yes, there is a lot of snake oil AI, and this will only increase. But these systems can and do work. I am sure there are AI experts building these systems right now.