A fundamental property of AI systems is that their propensities can be accurately measured and arbitrarily adjusted. Currently, human judges sentence women more leniently than men, which has various societal benefits (not breaking up families, especially). If you replace that with an "AI judge", it's no longer rational-minded judges applying their best judgment. The disparity can now be fixed in a single place, and affect every new convict across the country at once. You won't be able to have "sentencing guidelines", you'll have to have reproducible equations that always produce the same sentence for equivalent situations. Great! But now, that equation can be changed centrally. By a head of state trying to win an election, or the corporation making the AI, that wants to hype its brand (whether that means woke, or populist conservative).
There is a fundamental risk to AI being used at all, no matter how it's coded or regulated. I much prefer a flawed system where a few thousand people are at the mercy of one man's judgment (over that judge's career), to an even worse system where we're al cogs in an equation-machine, where the whole country is at the mercy of a corporate committee or regulatory agency's judgment.
You can't fix that by code, or by laws. Besides algorithmic bias researchers, who are all working to make better cogs, is anyone thinking about these more fundamental issues?