Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The NSA’s machine learning algorithm may be killing thousands of innocent people

arstechnica.co.uk

Post reply on HN
Print view View original

41–50 of 221 posts

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#41

Post by toomanybeersies »

nl wrote: post #9

This story is stupid. I'm sorry. No fan of the NSA, but the premise behind it is completely ridiculous. There is zero evidence of the repeatably asserted idea that the list this tool generates is any kind of kill list. It's a tool that generates indicators of people that may be worth looking at when trying to find couriers. That's a very specific subgroup of terrorists, and I find it entirely unsurprising that a jour…

> The NSA is many things, but "bad at Math" isn't one of them.

I agree, and with the pedigree of cryptology that has come from the NSA (and with alumni like Knuth), that's very well established.

But non-technical people tend to have a habit of taking technically brilliant (and often specific) things and using them in a way that they were never designed or intended.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#42

Post by leereeves »

avar wrote: post #39
nl wrote: post #9

This story is stupid. I'm sorry. No fan of the NSA, but the premise behind it is completely ridiculous. There is zero evidence of the repeatably asserted idea that the list this tool generates is any kind of kill list. It's a tool that generates indicators of people that may be worth looking at when trying to find couriers. That's a very specific subgroup of terrorists, and I find it entirely unsurprising that a jour…

Furthermore, even if the algorithm was generating kill lists directly there seems to be the implicit assumption that that would be worse than having some human "yes-men" who are directly beholden to orders of their immediate superiors and surely care more about their career advancement than bombing random Pakistanis in charge of the process. It's the same fallacy as being overly paranoid about the safety of self-driv…

...a good reason not to kill anyone without more than the decision of an AI or a council of human bureaucrats and politicians.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#43

Post by luisjgomez »

"Under the random selection of a tiny subset of less than 0.1 percent of the total population, the density of the social graph of the citizens is massively reduced, while the "terrorist" cluster remains strongly interconnected."

There is no indication in the slides that the feature calculations are done on the smaller subgraph. The above comment doesn't hold if the calculations are done on the entire graph and then the 100k are sampled for training.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#44

Post by Ntrails »

Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

>We can't even test it because there is no validation set and unreliable labels.

You can't test it, the NSA surely can and almost certainly can and did.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#45

Post by reitzensteinm »

nl wrote: post #9

This story is stupid. I'm sorry. No fan of the NSA, but the premise behind it is completely ridiculous. There is zero evidence of the repeatably asserted idea that the list this tool generates is any kind of kill list. It's a tool that generates indicators of people that may be worth looking at when trying to find couriers. That's a very specific subgroup of terrorists, and I find it entirely unsurprising that a jour…

> Also, criticizing the NSA on their knowledge of statistics seems unwise. The NSA is many things, but "bad at Math" isn't one of them.

Surely this would also apply to NASA, but then you go and read Feynman's analysis of the Challenger disaster and it doesn't seem at all the case.

I think the really unwise thing would be to ascribe abilities to an organization based on the abilities of individual members as politics starts to take over.

http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger...

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#46

Post by qeabc »

pavelrub wrote: post #30
Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

> The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. Not assassinate them without a trial. Which seems to be exactly the case. Ahmad Zaidan, for example, wasn't assassinated. In fact even if the algorithm was 100% correct - it makes no sense to assassinate people solely based on whether they are terrorists or not. A lot more can be gained from surve…

> So there is really nothing at all to suggest that those algorithms alone are used to create kill lists.

The first line of the article actually suggests this: 'In 2014, the former director of both the CIA and NSA proclaimed that "we kill people based on metadata."'

Also, even if something does not make sense does not mean that it does not seem to make sense to a particular person or organization.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#47

Post by cmdkeen »

Aeolos wrote: post #2

"Obliterated a wedding because people where celebrating by shooting in the air." This is the definition of evil. How and why is the Pakistani government allowing this program to operate within their territories?

Because most of the strikes are in the Tribal areas near the border with Afghanistan where the writ of the Pakistani state doesn't really run. The Pakistani military has received some bloody noses when trying to operate in the area. Even Wikipedia is happy to classify the whole thing as a war[0]. Over 6,000 members of the Pakistani security forces have been killed, 20,000 civilians and 35,000 "insurgents".

Pakistan gets significant military aid from the US to help put down an insurgency by groups that would overthrow the state. On a realpolitik level they also get to blame the US when a strike goes wrong whereas if they were dropping their own bombs they would have to put up with the blame. Mistakes always happen in war, I'd suggest however that a drone strike has significantly more time to conduct pre-launch checks than an F16 pilot does.

[0] https://en.wikipedia.org/wiki/War_in_North-West_Pakistan

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#48

Post by nl »

titzer wrote: post #33
pavelrub wrote: post #24

Earlier quoted context omitted.

> However we have no assurance that this tool was not used to generate kill targets without any human input. That's absurd. We also have no assurance that the NSA isn't communicating with aliens to generate kill lists of people who will oppose the alien invasion of 2021, though we do have common sense to filter out such absurd scenarios. "We have no assurance that this isn't so" isn't an argument.

I'll bite. The difference is that communicating with aliens requires positing the existence of aliens, whereas simply generating a kill list is already something we know they do, though we don't know how it works. It isn't "common sense" to filter out entirely obvious actions the NSA would take, and given the scale of disclosures from the Snowden files you ought to have already expanded your imagination away from "th…

The fact that 2,500 to 4,000 people have been killed in Pakistan according to a completely opaque process for classifying targets and blasting them from the sky--a process that operates entirely outside any law--ought to be considered absurd, shocking, frightening, and soul-crushingly inhuman. But it's not. We're arguing about with each other about bullshit.

I'm incredibly appalled by that. That's why stories like this are dreadful. They overreach in their conclusions, and will be easily denied by people involved in the programs who will then produce evidence to show that this particular program does exactly what it tries to to: identify terrorist couriers.

That denial and evidence will then discredit all the sensible arguments about the drone strike program.

Don't believe it? There's a discussion down-thread where someone is equating this list with the US Terrorism Watch Lists. Their clearly not the same thing at all (once glace at the slides shows you that), but they demand evidence. Of course, I can't show evidence that will convince them, but at some point an agency will, and they'll show exactly how the ist in this article (or some other list) is very accurate (I'm sure there is some list that is) and that will discredit the whole argument against the watch lists.

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#49

Post by anc84 »

flashman wrote: post #27
Houshalter wrote: post #7

They literally named it Skynet. They have an evil sense of humor. Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels. The reasona…

If you are going to put surveillance on potential terrorists, you should also be putting surveillance on people not suspected of being terrorists, so you can judge the rate at which your surveillance personnel falsely accuse people being terrorists based on having been selected for surveillance. Single or double blind that process.

They could put the surveillance on themselves for that good people control group. Oh wait...

Re: The NSA’s machine learning algorithm may be killing thousands of innocent people

#50

Post by venomsnake »

Aeolos wrote: post #2

"Obliterated a wedding because people where celebrating by shooting in the air." This is the definition of evil. How and why is the Pakistani government allowing this program to operate within their territories?

They could always not allow and be left to deal with Afgani collapse on their own. And with soaring and increasingly nationalistic India under Modi.
Post reply on HN