Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Suspicion Machine

wired.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: The Suspicion Machine

#11

Post by Biologist123 »

safety1st wrote: post #8
Biologist123 wrote: post #7

Earlier quoted context omitted.

In the west, I sometimes wonder if we already have a scoring system which ensures only good citizens can access housing, food and the rest. It’s called “money”, which together with credit scoring functions to make sure you abide by the rules society sets: no pay, no play.

This is like equating dick pic spam with sexual abuse, yes the first one is problematic, but no it is not equal to the second, in fact it's a disservice to victims of genuine abuse to equate them. I can get money from almost anybody who's willing to give it to me for any reason, the government's influence and visibility into this are limited. I can earn money in some other currency/jurisdiction and convert it, too. A…

Your analogy is a bit of an odd one, as unsolicited dick pics are undoubtedly a form of sexual abuse.

Re: The Suspicion Machine

#12

Post by thedailymail »

CommitSyn wrote: post #5
friend_and_foe wrote: post #2

https://archive.ph/9Ibjn

I desperately want to read this article. Anyone else getting /constant/ cloudflare CAPTCHAS while trying to access archive.ph? Android with mobile Firefox or chrome, first with wifi then mobile then VPN. No matter what I can't access it and it won't stop showing the "are you a human?" check page, even after trying the CAPTCHA the first time it stops showing. Must be a cloudflare outage?

If you are using Cloudflare 1.1.1.1, try switching it off or using another DNS resolver. There are reported issues with archive.md blocking access from 1.1.1.1

Re: The Suspicion Machine

#13

Post by lozenge »

Isn't the Dutch language requirement, which is codified as an eligibility criteria, already intended to create an underclass of residents?

I think it is morally justifiable as a residency requirement, but not justifiable to let people live there without being able to receive government support.

I think it's a situation where the government want to be racist or at least xenophobic, the citizens agree, but the law prevents them. Accenture was drafted in to get around the law.

Re: The Suspicion Machine

#15

Post by lozenge »

Biologist123 wrote: post #7
rnk wrote: post #3

This is a real problem. These algorithms are a way for us in the west to experience social credit type scores like we read about from China. I'm sure there's someone here who was unfortunate to have a name that overlapped in some way with an identified "terrorist". Don't forget that when you buy an airplane ticket, there's that always slightly worrisome option to "add your special id number if you are incorrectly lis…

In the west, I sometimes wonder if we already have a scoring system which ensures only good citizens can access housing, food and the rest. It’s called “money”, which together with credit scoring functions to make sure you abide by the rules society sets: no pay, no play.

Money in the US traces back to redlining, racist deeds, racist lending criteria, white areas enforced with mob violence etc. Money is just the modern legal way of carrying over past discrimination.

Re: The Suspicion Machine

#16

Post by RandomLensman »

lozenge wrote: post #15
Biologist123 wrote: post #7

Earlier quoted context omitted.

In the west, I sometimes wonder if we already have a scoring system which ensures only good citizens can access housing, food and the rest. It’s called “money”, which together with credit scoring functions to make sure you abide by the rules society sets: no pay, no play.

Money in the US traces back to redlining, racist deeds, racist lending criteria, white areas enforced with mob violence etc. Money is just the modern legal way of carrying over past discrimination.

Interesting that money in the US should have a very different origin - how did that happen?

Re: The Suspicion Machine

#17

Post by 0xDEAFBEAD »

lozenge wrote: post #13

Isn't the Dutch language requirement, which is codified as an eligibility criteria, already intended to create an underclass of residents? I think it is morally justifiable as a residency requirement, but not justifiable to let people live there without being able to receive government support. I think it's a situation where the government want to be racist or at least xenophobic, the citizens agree, but the law prev…

>I think it is morally justifiable as a residency requirement, but not justifiable to let people live there without being able to receive government support.

Given the choice between accepting a small number of immigrants with government support, or a large number of immigrants without government support, the second option seems more humane to me.

By choosing the first option, you are effectively creating a privileged class based on whatever criteria your country uses to accept immigrants. The underclass might be out of your sight, but it still exists.

As long as you are using some criteria to accept immigrants, "willing to work without government support" (at least until they become fluent in the local language) seems like a perfectly reasonable criterion to me. And it is a criterion that gives your nation the capacity to accept a larger number of migrants without breaking the budget -- thereby helping more people from developing countries improve their economic situation.

Re: The Suspicion Machine

#18

Post by 0xDEAFBEAD »

rnk wrote: post #3

This is a real problem. These algorithms are a way for us in the west to experience social credit type scores like we read about from China. I'm sure there's someone here who was unfortunate to have a name that overlapped in some way with an identified "terrorist". Don't forget that when you buy an airplane ticket, there's that always slightly worrisome option to "add your special id number if you are incorrectly lis…

>The inability to sue to identify the problem or correct it is a real loss of autonomy and freedom.

It's unclear if "inability to sue" applies in this case. For example:

>Discriminating against people based on their gender and ethnic background is illegal in the Netherlands, but converting that information into data points to be fed into an algorithm may not be. This gray area, in part the result of broad legal leeway granted in the name of fighting welfare fraud, lets officials process and profile welfare recipients based on sensitive characteristics in ways that would otherwise be illegal. Dutch courts are currently reviewing the issue.

Re: The Suspicion Machine

#19

Post by 0xDEAFBEAD »

The algorithm described in this article seems very bad. But I would argue that ML risk scores can, in principle, be better than human judgment.

Humans seem more subject to bias than algorithms are. Algorithms only look at data, but humans are additionally vulnerable to stereotypes and prejudices from society.

Furthermore, using an algorithm gives voters an opportunity to have a debate regarding how best to approach a problem like welfare fraud.

Human judgment relies on bureaucrats who are often biased and unaccountable. It's infeasible for voters to audit every decision made by a human bureaucrat. Replacing the bureaucrat with an algorithm and inviting voters to audit the algorithm seems a heck of a lot more feasible.

I give the city of Rotterdam a lot of credit for the level of transparency they demonstrated in this article. If they want to be successful with algorithmic risk scores, I think they should increase the level of transparency even further. Run an open contest to develop algorithms for spotting welfare fraud. Give citizens or representatives information about the performance characteristics of various algorithms, and let them vote for the algorithm they want.

In the same way politicians periodically come up for re-election, algorithms should periodically come up for re-election too. Inform voters how the current algorithm has been performing, and give them the option to switch to something different.

Re: The Suspicion Machine

#20

Post by 0wis »

I am not sure the data model is the problem here. I feel like the journalist tried really hard to make a case against scoring (which I do not like either), but overlooked the fact that the whole system in which its embedded is bad. The case should not be on the technology but on its usage.

It’s already looking like a bad journalism piece in the first part :

” Being flagged for investigation can ruin someone’s life, and the opacity of the system makes it nearly impossible to challenge being selected for an investigation, let alone stop one that’s already underway. One mother put under investigation in Rotterdam faced a raid from fraud controllers who rifled through her laundry, counted toothbrushes, and asked intimate questions about her life in front of her children.”

Here the problem is not the algorithm, its the investigators.

Another ethical problem for me : the system of flagging in whole relied partly on anonymous tips from neighbors. I am not an expert but I feel more at ease about a system that rely on a selection algorithm + randomness than delation.

I think the problem was the processes around the algorithm not its existence itself. The journalist seems to assume during the whole piece that the algorithm will become the main/only way to identify fraudsters. If its the case, it’s terribly wrong because how are you training your algorithm then ?

Most of the time, the piece try to put the reader in an emotional state of fear and anger and is not at all doing any analysis, while faking it using a lot of numbers and graphs.

Sorry for the long rant but I am surprised that this came from Wired which I consider quite good on tech topics, and that its on HN 2nd page.

I am against government scoring and algorithms for legal / police cases precisely because it can be badly used by powerful people.

Am I the only one to feel that its not a good article ?

Post reply on HN