Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The NSA’s machine learning algorithm may be killing thousands of innocent people

arstechnica.co.uk

Post reply on HN
Print view View original

61–70 of 221 posts

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#61

Post by Vivtek »

spazmaster wrote: post #22

The thing that stood out for me is "Somewhere between 2,500 and 4,000 people have been killed by drone strikes in Pakistan since 2004". WHAT!? That is so wrong! This stinks and we're making fuss about mathematics. Just read that sentence again.

We're making a fuss about mathematics because this program is sold on the basis of the infallibility of mathematical tools. Especially for non-technical people of a certain age, this is a very persuasive argument (just watch any movie with a computer in it made in the 60's or 70's - computers are magical instruments that find truth where mere humans cannot).

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#62

Post by qeabc »

Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

Perhaps someone who knows the topic can say whether this mathematical phenomena of "adversarial examples" for neural networks can be translated to random forests: http://www.kdnuggets.com/2015/07/deep-learning-adversarial-e...

Barely perceptible changes to an image cause it to be misclassified by neural networks which never saw the image before (this is important, because it rules out simple overfitting). As a non-expert, this suggests to me that, at least for some algorithms, the reasoning "statistical algorithm estimates 99.999% probability of guilt" implies "guilty beyond a reasonable doubt" is unreliable at best.

There is great potential here for safe, effective government if machine learning output is only ever input for humans with common sense, life experience and the ability to interact with the world.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#63

Post by akerro »

agd wrote: post #13

Big data analysis + mass surveillance is a frightening prospect. Of course you can train software to look for 'terrorists', but you could also train it to look for: - whistleblowers - minority groups (e.g. gay people, particular religious beliefs, political affiliation) - political dissidents - journalists whose behaviour changes - personal vulnerabilities (affairs, mental health issues etc) Think what authoritarian…

- political dissidents

https://en.wikipedia.org/wiki/McCarthyism

And that's what it is used for. Australian gov. used their abilities to mark people who say against coal energy as anti-government, all they did was support renewable sources of energy. https://overland.org.au/2014/07/surveillance-of-activists-is...

and Canada: http://www.theguardian.com/environment/2013/feb/14/canada-en...

- minority groups

FBI was spying on black-rights activists, since ever. https://www.eff.org/deeplinks/2014/11/fbis-suicide-letter-dr...

- whistleblowers

No example needed.

- journalists whose behaviour changes

https://www.opendemocracy.net/ourkingdom/duncan-campbell/gch...

Campbell in '70s discovered ECHELON, NSA admitted he was their personal target since then.

- personal vulnerabilities (affairs, mental health issues etc)

https://en.wikipedia.org/wiki/SEXINT https://en.wikipedia.org/wiki/LOVEINT

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#64

Post by scotty79 »

> ... possibly resulting in their untimely demise.

I get increasingly annoyed by these kinds of euphemisms in articles discussing US actions. As if it was something humorous.

Should be something like: ... possibly resulting in them getting assassinated by US military.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#67

Post by oafitupa »

Aeolos wrote: post #58
spazmaster wrote: post #22

The thing that stood out for me is "Somewhere between 2,500 and 4,000 people have been killed by drone strikes in Pakistan since 2004". WHAT!? That is so wrong! This stinks and we're making fuss about mathematics. Just read that sentence again.

Exactly. In which freaking world are assassinations without due process legal or even desirable? The mere fact that we are debating whether someone was or wasn't assassinated is a perversion in itself. Arrest them and try them in a court of justice. End of story.

Arrest them? They were in freaking Pakistan. Why would you arrest them? You guys are going crazy. Don't be surprised when you travel and realize the world hates you.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#68

Post by mcv »

cmdkeen wrote: post #47
Aeolos wrote: post #2

"Obliterated a wedding because people where celebrating by shooting in the air." This is the definition of evil. How and why is the Pakistani government allowing this program to operate within their territories?

Because most of the strikes are in the Tribal areas near the border with Afghanistan where the writ of the Pakistani state doesn't really run. The Pakistani military has received some bloody noses when trying to operate in the area. Even Wikipedia is happy to classify the whole thing as a war[0]. Over 6,000 members of the Pakistani security forces have been killed, 20,000 civilians and 35,000 "insurgents". Pakistan g…

But the end result of all that is just that it generates even more hostility towards the US and feeds terrorism. The US is creating its own enemies this way. Again.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#70

Post by appleflaxen »

The founding fathers put due process into the constitution / bill of rights because it's a fundamental check on government bureaucracy run amok, and prevents physical harm from coming to people as the result of a capricious executive whim.

Now, we learn that the executive has excused themselves from following this process (which applies to Americans, some of which these victims may be, even when they are not in America).

The farther open we pull the NSA lid, the more revolting discoveries we find.

The individuals behind these decisions should be identified, and tried in the court of law.

Post reply on HN