Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The NSA’s machine learning algorithm may be killing thousands of innocent people

arstechnica.co.uk

Post reply on HN
Print view View original

51–60 of 221 posts

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#51

Post by pmichaud »

Sir_Substance wrote: post #38
Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

>The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. That is in no way a reasonable thing to do if you're only 50% confident in your results. A reasonable thing to do would be scrap the whole system and make something more reliable than a coin flip.

Characterizing it as a coin flip is unfair I think. If this system can promote individuals to attention out of millions of targets, telling us that this person out of all the millions has a 50/50 chance of being a terrorist, then that is hugely valuable information that deserves further inquiry.

Now, of course we don't know how good the rate is or what other evidence they already have or what exactly a terrorist is for that matter. But in principle, you'd be throwing out a huge amount of evidence if got rid of a system that changed a target's chance from 1 in a million to 1 in 2.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#52

Post by facepalm »

agd wrote: post #13

Big data analysis + mass surveillance is a frightening prospect. Of course you can train software to look for 'terrorists', but you could also train it to look for: - whistleblowers - minority groups (e.g. gay people, particular religious beliefs, political affiliation) - political dissidents - journalists whose behaviour changes - personal vulnerabilities (affairs, mental health issues etc) Think what authoritarian…

It seems like a safe bet that they will try to learn as much about people as possible. Not so different from classifying them into people who are likely to buy a new car or nutritional supplements.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#53

Post by nl »

Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. Not assassinate them without a trial.

This appears to be what they do?

The National Security Agency reportedly tracked phone calls between the courier Abu Ahmed al-Kuwaiti's relatives in the Persian Gulf to all numbers in Pakistan, and NSA surveillance eventually tracked Abu Ahmed al-Kuwaiti's location in Pakistan via one such phone call.... In August 2010 they tracked al-Kuwaiti as he drove from Peshawar to a residence in Abbottabad – and as analysts inventoried the compound's striking security features they became convinced that it housed a high-level al-Qaeda figure... In September 2010, the CIA concluded that the compound was "custom built to hide someone of significance" and that it was very likely that Osama bin Laden was residing there[1]

As it turned out, that was where bin Laden was killed, so that process seemed to have worked.

I'd note that the arstechnica article gives no evidence at all that this particular program is used to generate kill lists. As you note, identifying possible couriers and just killing them without further investigation is a dumb idea. Fortunately I don't see any evidence in this or other articles that this is what happens.

[1] https://en.wikipedia.org/wiki/Abu_Ahmed_al-Kuwaiti

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#55

Post by monort »

nl wrote: post #9

This story is stupid. I'm sorry. No fan of the NSA, but the premise behind it is completely ridiculous. There is zero evidence of the repeatably asserted idea that the list this tool generates is any kind of kill list. It's a tool that generates indicators of people that may be worth looking at when trying to find couriers. That's a very specific subgroup of terrorists, and I find it entirely unsurprising that a jour…

We know that they do strikes based on circumstantial evidences (signature strikes):

http://www.theatlantic.com/international/archive/2015/04/war...

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#57

Post by purpled_haze »

Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

Just don't try to shut it down.

"We love you, Skynet! We just need to do some routine maintenance."

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#58

Post by Aeolos »

spazmaster wrote: post #22

The thing that stood out for me is "Somewhere between 2,500 and 4,000 people have been killed by drone strikes in Pakistan since 2004". WHAT!? That is so wrong! This stinks and we're making fuss about mathematics. Just read that sentence again.

Exactly.

In which freaking world are assassinations without due process legal or even desirable? The mere fact that we are debating whether someone was or wasn't assassinated is a perversion in itself.

Arrest them and try them in a court of justice. End of story.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#60

Post by edge17 »

Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

Many other countries have 'Skynet' programs https://en.m.wikipedia.org/wiki/Skynet
Post reply on HN