Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Courts are using risk-assessment software to sentence criminals

wired.com

Post reply on HN
Print view View original

61–70 of 90 posts

Re: Courts are using risk-assessment software to sentence criminals

#61

Post by sidlls »

arprocter wrote: post #20
rayiner wrote: post #8

> These algorithmic outputs inform decisions about bail, sentencing, and parole. Each tool aspires to improve on the accuracy of human decision-making that allows for a better allocation of finite resources. It's really not clear to me that much is gained from having very precise decisions made about bail and sentencing. Trying to predict the future is a fool's errand, whether a judge does it or a computer. It'd be b…

Devil's advocate, but the system in the article sounds similar to an insurance company calculating risk. Perhaps the real objection is the lack of transparency behind the calculations?

In that case the insurance companies' models are as suspect as these criminal risk models.

Re: Courts are using risk-assessment software to sentence criminals

#62

Post by humanrebar »

hackuser wrote: post #56
humanrebar wrote: post #52

Earlier quoted context omitted.

> ...other correlated factors will allow the algorithm to continue to enforce this racism, but now with legal immunity. I'm sympathetic, but how do we keep from causing more harm than good here? It seems like blindly ignoring all data that correlates with race is also bad. We do this in some circumstances already (i.e., IQ tests when doing hiring or admissions) and I think it's a defensible point that we're worse off…

> blindly ignoring all data that correlates with race is also bad This is a very theoretical, futuristic red herring. Race factors into decision-making far, far too much. It's like worrying that if someone developed a perfect, bug-free crypto library, then everyone would use it and we'd have a monoculture - not really a present concern, and in fact a good problem to have (nor is it an eloquent analogy, but I have to…

I can't really tell if you are for or against algorithmic sentencing from this post.

I think there are fair existing examples both for and against it, so I'm not sure calling either position futuristic or theoretical is fair.

Re: Courts are using risk-assessment software to sentence criminals

#63

Post by jpab »

mcherm wrote: post #29

One of my big concerns is whether the algorithms are institutionalizing racism, with legal decisions that make it impossible to challenge these. After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't fooli…

I agree wholeheartedly with this concern, but I think there is also a counter-argument too. If you assume we're starting from a situation where there is bias, then you probably want to look for ways to reduce that bias. The software may have an advantage here too.

It may be easier to objectively measure and correct for undesired (racial, or otherwise) bias in an automated system than it is to objectively measure and correct for bias in human judgement. And even if it's more difficult to correct for undesired bias in an automated system, at least that correction is more likely to be permanent and applied uniformly, compared to things like giving training courses for humans, which will vary in effectiveness across the recipients of the course, and will need to be refreshed frequently.

There are a lot of qualifications and "maybe"s in that argument though.

Re: Courts are using risk-assessment software to sentence criminals

#64

Post by JustSomeNobody »

mcherm wrote: post #29

One of my big concerns is whether the algorithms are institutionalizing racism, with legal decisions that make it impossible to challenge these. After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't fooli…

So, when we train all the AI and robots that are going to take over all of our jobs in the next 5 years, we need to make sure we are very careful.

Re: Courts are using risk-assessment software to sentence criminals

#66

Post by mproy »

vmarsy wrote: post #50
mproy wrote: post #46

Earlier quoted context omitted.

" ... other correlated factors will allow the algorithm to continue to enforce this racism" Which correlated factors are you thinking of?

Even if you are careful to remove all features you think could tell the race (let's say geographic location etc), it could still be giving harsher sentences to minorities if those are treated differently upfront. For instance, it would make sense for the model to learn that the more charges count the harsher the sentence. This sounds right: a bank robbery should be less punished than a bank robbery + a carjacking, at…

But if all of that is true, the model doesn't change the equation. The problems you outline need to be addressed upstream. Trying to account for and correct them at the point of sentencing is very problematic.

Re: Courts are using risk-assessment software to sentence criminals

#67

Post by mproy »

advisedwang wrote: post #49
mproy wrote: post #46

Earlier quoted context omitted.

" ... other correlated factors will allow the algorithm to continue to enforce this racism" Which correlated factors are you thinking of?

Some proxies for race: - Where somebody lives - Their name - What sports teams somebody supports - Their finances - consumer preferences - education level - what medical conditions somebody has - political leanings I'm not saying these factors can't legitimately be used in a risk assessment, but they could be used to make a good bet on race.

I don't see how those factors are relevant at all, regardless of the race of the criminal. The problem to address is the lack of transparency. Trying to guess what inputs are being factored in is pointless.

Re: Courts are using risk-assessment software to sentence criminals

#68

Post by hackuser »

bmh_ca wrote: post #9

I wonder if the algorithm is evidence-based, and learns from the results of prior decisions. In which case it is possible it would eventually discover that in the USA incarceration is very strongly linked to recidivism. It follows that the algorithm might refuse to incarcerate many convicts. Which is arguably exactly what the algorithm should do, namely what politicians will/can not: employ evidence to advance the me…

> what the algorithm should do, namely what politicians will/can not

It's dangerous to imagine that the algorithms won't be used just as politically as anything else. The difference is, instead of a mostly transparent process in a legislature, the decisions will be made between powerful people and software developers.

Re: Courts are using risk-assessment software to sentence criminals

#69

Post by HillaryBriss »

kpil wrote: post #48
HillaryBriss wrote: post #11

Earlier quoted context omitted.

> It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment it may not be better, but, isn't it more predictable ? also, doesn't it protect a judge and the justice system against charges of favoritism, discrimination, and other kinds of bias? i mean, i'm not a lawyer and i truly don't know ... but weren'…

But the point is that it's fake predictability, as you can't make arbitrarily complex rules and hope to have built the Totally Fair System. (Provably not, actually.) The best thing we can do is to admit it's not going to be perfect, and use our own reasoning and feelings, and make sure the process is transparent and the people are held accountable.

> it's fake predictability

so, a criminal defendant and their lawyer have no better chance of predicting the sentence handed down given guidelines than they would if there were no guidelines?

Re: Courts are using risk-assessment software to sentence criminals

#70

Post by bbig3831 »

mcherm wrote: post #29

One of my big concerns is whether the algorithms are institutionalizing racism, with legal decisions that make it impossible to challenge these. After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't fooli…

There was a Washington Post article and arXiv paper on this topic several months ago. [1][2] The problem boils down to the definition of bias. Northpointe, the company that owns COMPAS, says the algorithm treats black and white defendants equally, which is true. Others claim that its predictions for blacks are wrong more often than for whites, which is also true. The main point of the paper is that you can't have your cake and eat it too. If you don't want to consider race, then the predictions will be wrong more often for blacks. If you don't want the predictions to be wrong more often for blacks, the algorithm has to consider race. (I'd recommend the articles for a better explanation.)

In my mind, the ownership of the software is what's troubling. First, it's not public, so nobody can look at the guts. And second, the Loomis decision constructs even more walls around the software. It's hard for people to critically examine and change something they can't see.

[1] https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... [2] https://arxiv.org/pdf/1609.05807.pdf

Post reply on HN