Live data from Hacker News

XKeyscore: NSA program collects 'nearly everything a user does on the internet'

theguardian.com

11–20 of 641 posts

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#11
One of the screen shots:

http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video...

says: Top Secret Comm(?) REL() to USA, AUS, CAN, GBR, NZL

confirming the previous suspicions that many other governments are on board.

Der Spiegel actually has reported a few weeks back about XKeyscore [1] and that it is used by the BND (Germany's NSA). I.e. all this data is also available to the NSA equivalents of Australia, Candana, Great Britain and New Zealand.

Many Americans trust their government (unfortunately), will they also trust the other governments?

[1]:

http://www.spiegel.de/international/world/german-intelligenc...

http://www.spiegel.de/international/germany/german-intellige...

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#12
This is brilliant, I love the screenshots:

Foreignness factor:

The person has stated that he is located outside the U.S.

Human intelligence source indicates person is located outside the U.s.

The person is a user of storage media seized outside the U.s.

Foreign govt indicates that the person is located outside the U.s.

Phone number country code indicates the person is located outside the U.s.

Phone number is registered in a country other than the U.S.

SIGINT reporting confirms person is located outside the U.S.

Open source information indicates person is located outside the U.s.

Network, machine or tech info indicates person is located outside the U.s.

In direct contact w/ tgt overseas no info to show proposed tgt in U.S.

It's quite easy to lose the protections of a U.S. citizen indeed!

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#13
Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595.

Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595

Which leads us to the Facebook profile (https://www.facebook.com/arash.gorjipour.5) of an individual, real or contrived, named "Arash Gorjipour". His email address and phone number are all exposed in one of his uploaded photos: http://i.imgur.com/0UUk5cB.jpg

I wonder what the reason for this man being in these slides is.

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#15
At what point do the mathematical limits of data mining kick in here? How useful is all this information?

I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe.

As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of misses and the minimum number of false positives. My impression is that this becomes progressively harder-- the golden region between too many false positives and too many false negatives becomes smaller and smaller and harder to hit.

Eventually you either miss important hits, namely the next terrorist attack, or you get swamped with false positives that you have to manually investigate and rule out.

I'd love someone who does know more here to chip in, but my personal suspicion is that this actually has a pretty huge pork angle to it. How much money are the contractors getting for building this stuff?

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#16
post #7

I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques. Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.

Hadoop[0] and OpenStack[1]

I think the era of government being far ahead of commercial tech capability is over. The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors.

[0] http://online.wsj.com/article/SB1000142412788732349560457853...

[1] http://www.youtube.com/watch?v=NgahKksMZis

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#17

Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595. Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595 Which leads us to the Facebook profile ( https://www.facebook.com/arash.gorjipour.5 ) of an individual, real or contrived, named "Arash Gorjipour". His email address and p…

Probably picked at random. The screenshot 'test case' could have been you. We are all in the database after all.

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#18

Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595. Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595 Which leads us to the Facebook profile ( https://www.facebook.com/arash.gorjipour.5 ) of an individual, real or contrived, named "Arash Gorjipour". His email address and p…

They may have used an existing public profile since he's already displayed it openly. He's a realestate broker after all so, presumably, he's got "nothing to hide".

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#19
post #15

At what point do the mathematical limits of data mining kick in here? How useful is all this information? I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe. As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of mis…

From the article

The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."

Re: XKeyscore: NSA program collects 'nearly everything a user does on the internet'

#20
post #6

Imagine if storage limitations weren't holding back the NSA. Those 60TB density HAMR[1] drives that are due in 2016 are really going to take invasive to a whole new level. [1] http://storageeffect.media.seagate.com/files/2012/03/perpham...

20 TB per day is nothing these days.
Post reply on HN