Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sentenced by Algorithm

nybooks.com

Post reply on HN
Print view View original

31–40 of 61 posts

Re: Sentenced by Algorithm

#31

Post by rincebrain »

nickthemagicman wrote: post #22
ZephyrBlu wrote: post #20

Earlier quoted context omitted.

I suspect the "may have" is what set people off.

Thanks for letting me know. So strange. The 'algorithm' is the noun object of the preposition. Not society or history . Is modern English education really that poor? Or is there purposeful misinterpretation in order to be offended? I just don't get it.

When I read it, as a native speaker, I read "treated poorly by the algorithm" as being a term for the behavior of culture and systems of the time, and not a hypothetical algorithm from the future that took as input the culture of the time and made decisions about fairness.

I think it was the use of "the algorithm" and not "an algorithm" that most affected my reading there - "poorly by an algorithm" comes across, to me, as "poorly by some algorithm", rather than "poorly by _the_ algorithm", implying a specific instance of a thing, already known by the reader or referred back to later, doing it.

Consider "treated poorly by the system" versus "treated poorly by such a system", which has an even clearer connotation, to me.

Re: Sentenced by Algorithm

#32

Post by noduerme »

rincebrain wrote: post #31
nickthemagicman wrote: post #22

Earlier quoted context omitted.

Thanks for letting me know. So strange. The 'algorithm' is the noun object of the preposition. Not society or history . Is modern English education really that poor? Or is there purposeful misinterpretation in order to be offended? I just don't get it.

When I read it, as a native speaker, I read "treated poorly by the algorithm" as being a term for the behavior of culture and systems of the time, and not a hypothetical algorithm from the future that took as input the culture of the time and made decisions about fairness. I think it was the use of "the algorithm" and not "an algorithm" that most affected my reading there - "poorly by an algorithm" comes across, to m…

My first reading as a native speaker was that "the algorithm" meant "the Man". But after several readings, I think he meant that this algorithm would categorize someone who was legally considered less than a citizen as a more likely recidivist. Which is an interesting point. Particularly in the US where there's a notion of illegal immigrants having already broken the law. Would this algorithm consider them to be more likely to be repeat offenders if it were weighted to treat noncitizenship as a factor, or overstaying a visa as a crime? And if so, then why wouldn't it weight slaves as more likely to reoffend. And if it did that, what would stop it from weighting the descendants of slaves the aame way?

Re: Sentenced by Algorithm

#33

Post by noduerme »

masswerk wrote: post #25
onos wrote: post #3

It seems the appropriate method to judge this approach is in comparison to the baseline referenced at the start: humans making “black box” judgement calls that are subject to the individual judge’s biases. If the computer programmed decision making were (already is?) transparent, then we could critique it and work to ensure something resembling fair is encoded.

I can actually see a use case in earmarking court decisions for review that are far off from what may have been expected. Handing over the decision altogether, however, is like handing justice over to a theatrical magician ("The Great Fully Automated Barnum Court"): it's commercially incentivized, you may not know the trick, because it's a trade secret, and it's not the real thing, either. (But, exactly because of th…

This reminded me of something I read, and I can't recall whether it was in Argentina under the junta, or in North Korea, or in the USSR ...but the gist was that people sentenced for minor crimes or for irritating the state were essentially assigned random sentences and punishments, divorced from or even absurdly light or harsh relative to their suppose crimes, as a way of terrorizing the population.

Or maybe I'm thinking of a Kafka story, or Dostoevsky's mock execution. Or 1984. Shit. I've lost it.

[edit] Might have been from the Killing Fields. Wow, this is really dredging up a lot of wonderful historical references...

Re: Sentenced by Algorithm

#34

Post by unishark »

noduerme wrote: post #32
rincebrain wrote: post #31

Earlier quoted context omitted.

When I read it, as a native speaker, I read "treated poorly by the algorithm" as being a term for the behavior of culture and systems of the time, and not a hypothetical algorithm from the future that took as input the culture of the time and made decisions about fairness. I think it was the use of "the algorithm" and not "an algorithm" that most affected my reading there - "poorly by an algorithm" comes across, to m…

My first reading as a native speaker was that "the algorithm" meant "the Man". But after several readings, I think he meant that this algorithm would categorize someone who was legally considered less than a citizen as a more likely recidivist. Which is an interesting point. Particularly in the US where there's a notion of illegal immigrants having already broken the law. Would this algorithm consider them to be more…

I thought the US was somewhat unusual in its lax treatment of illegal immigration. However lying to or sneaking past border officials to enter a country is literally a crime yes (certainly for foreign nationals, and probably not kosher for citizens to do either). Though entering on a visitor visa and overstaying is just a civil violation with a civil penalty (deportation). If a citizen had a record of using false identity documents or trying to evade legally-enforced border crossings, a fair algorithm would presumably have to take that into account just as well.

Re: Sentenced by Algorithm

#36

Post by crooked-v »

noduerme wrote: post #33
masswerk wrote: post #25

Earlier quoted context omitted.

I can actually see a use case in earmarking court decisions for review that are far off from what may have been expected. Handing over the decision altogether, however, is like handing justice over to a theatrical magician ("The Great Fully Automated Barnum Court"): it's commercially incentivized, you may not know the trick, because it's a trade secret, and it's not the real thing, either. (But, exactly because of th…

This reminded me of something I read, and I can't recall whether it was in Argentina under the junta, or in North Korea, or in the USSR ...but the gist was that people sentenced for minor crimes or for irritating the state were essentially assigned random sentences and punishments, divorced from or even absurdly light or harsh relative to their suppose crimes, as a way of terrorizing the population. Or maybe I'm thin…

Don't forget the film Brazil, where a bug-induced misprinted warrant leads to an obviously wrong person being arrested and then interrogated to death, because how could the machine be wrong?

Re: Sentenced by Algorithm

#37

Post by foxfired »

Is the sentencing the bottleneck in the justice system? Is it something that is so tedious to do that we rather have a machine do it?

Or is it that we want to eliminate bias so we let the algorithm do it? Because machines are cold calculator that only cares about the data.

Or is it shifting the responsibility? If a judge gives a light sentence to a person who ends up committing a crime again, does the judge get the blame? Is the algorithm an externalizer that dilutes the responsibility? "Well the machine said he was clean, it knows a thing or two we don't know. Plus it's a black box I'm just an ol judge. I don't know how email works, but it works all the same."

Re: Sentenced by Algorithm

#38

Post by tdeck »

crooked-v wrote: post #35
jeffypoo wrote: post #30

Earlier quoted context omitted.

Your outrage radar is a little too sensitive pal.

You don't see any problem with the "may have"?

I think what OP meant is "if the algorithm were designed (or perhaps implemented) in the 1700s according to standards of fairness held by those in power at that time, then it might have treated African Americans unfairly (despite being an algorithm)". I'm not a mind reader but that seems to make some sense given the context of the discussion.

Re: Sentenced by Algorithm

#39

Post by TanteTara »

I miss any discussion about the elephant in the room: On what data does a potential algorithm base its decision? Who puts in that data?

If you know how the algorithm works (and this one must be open source), you can game it by manipulating the data it gets to see (or not) and how the specific wording is.

Can the algorithm generate and ask questions by itself? Again that would be pointless if you know the algorithm - you would know the question beforehand and how exactly a given answer influences the outcome.

This idea cannot ever work until you have a strong general AI (which you can not predict any more and which may have its own biases) and we are as far from a strong general AI as we have ever been.

Re: Sentenced by Algorithm

#40

Post by AstralStorm »

swayvil wrote: post #29
failuser wrote: post #28

Earlier quoted context omitted.

Happy? People sentenced by a judge or jury have a hope of appeal. People sentenced by Google don’t have any hope.

Yeah, but we would have a lot more free time. Life would be simpler. And as long as you keep your behavior within the statistical sweet-zone the justicebots will leave you alone. (Much like social media, actually)

Look up chilling effects. This would be literally a kind of enforced conservatism.

Imagine this was up when Gallileo was being sentenced. How many Socrateses do you want to prevent? And so on.

Society can be wrong and mindlessly enforcing status quo is often immoral.

Post reply on HN