AI is sending people to jail and getting it wrong

Articles I read about the failings of AI can generally be summarised as follows: Someone made a shitty statistical analysis and found out that the result was shitty when the shitty model was run on a perfectly adequate computer with shitty data. In other acronyms: SISO and also GIGO.

Re: AI is sending people to jail and getting it wrong

#5

Post by bko » Fri, Jan 25, 2019, 12:15 PM UTC

> So if you feed it historical crime data, it will pick out the patterns associated with crime. But those patterns are statistical correlations—nowhere near the same as causations. If an algorithm found, for example, that low income was correlated with high recidivism, it would leave you none the wiser about whether low income actually caused crime. But this is precisely what risk assessment tools do: they turn correlative insights into causal scoring mechanisms.

My main problem with this sort or argument is that it compares an algorithm to some ideal. As though a judge has some infallible insight from carefully studying empirical results for recidivism.

In regards to correlation vs causation, they're important when trying to prescribe solutions. For instance, suppose that higher rates of recidivism are caused by growing up in an abusive household. And having a violent parent also causes a lower income. Correlation vs causation is important when trying to remedy the problem. Giving the family money will not solve the underlying problem, just a side effect. Solving the side effect may still be worth it though. But if you're determining rates of recidivism, it would be better to look at the underlying cause (did you grow up in a violent household) rather than a side effect (poverty), but they would both lead to the same final recidivism score. And you can still reject both under the premise that the past that's out of your control should not be considered.

But even if you are set on only using the cause, statistical methods are more likely to find these results. And you can easily control what input is provided. You can tell a judge to ignore certain characteristics, but her actual reasoning process is likely opaque.

Re: AI is sending people to jail and getting it wrong

#6

Post by carl-erwin » Fri, Jan 25, 2019, 12:31 PM UTC

pmontra wrote: post #2

The site is down now, cached copy at https://webcache.googleusercontent.com/search?q=cache:EqbA9Y...

wrong link missing www in url.

https://www.technologyreview.com/s/612775/algorithms-crimina...

Re: AI is sending people to jail and getting it wrong

#7

Post by pmontra » Fri, Jan 25, 2019, 3:37 PM UTC

carl-erwin wrote: post #6

pmontra wrote: post #2

The site is down now, cached copy at https://webcache.googleusercontent.com/search?q=cache:EqbA9Y...

wrong link missing www in url. https://www.technologyreview.com/s/612775/algorithms-crimina...

Wow, somebody still doesn't handle both the www and the bare domain name URLs. I didn't think about it. Thanks.

Re: AI is sending people to jail and getting it wrong

#8

Post by kstenerud » Fri, Jan 25, 2019, 7:13 PM UTC

The problem is far more fundamental. America operates under the premise that crime is a moral failing, and that reducing crime is mostly a matter of strong enough deterrents.

The underlying assumption that logically follows this thinking is that criminals are by choice beyond redemption (essentially enemies of society), and so your best course of action is separating them from the rest of the population. What happens after that is not your concern: out of sight, out of mind.

This mindset is readily apparent in the almost complete lack of societal reintegration services, and the casual use of isolation, despite the ample evidence of how much damage this does.

Re: AI is sending people to jail and getting it wrong

#9

Post by dczx » Fri, Jan 25, 2019, 7:45 PM UTC

The first concern is the vanishing/exploding gradient issue within neural network based models and the inherent bias.

But greater is the propensity for abuse, the algorithm that determines the risk model isn't publicly accessible.

Once we get to a point where an AI controls sentencing, it could be manipulated. The idea that an AI get's to determine which humans go to jail is the last thing we should ever utilize AI for.

A more important issue would be to focus on prison reform, let's reduce the causation of crime. Unfortunately private prisons are expanding right now and there's no reason to assume the prison lobby wouldn't love to influence an algorithm designed to send more people to jail.