Ethics Aside, Is NSA's Spy Tool Efficient?

What a world we live in! :(

But no, it isn't, because it can be gamed like any other rules system. The NSA probably thinks their system hasn't been explored yet, relying as they do primarily on security through obfuscation.

One of the greatest pleasures in life is examining "black box" systems and figuring them out. The NSA would be fools to expect that their system is not already being gamed.

Think about the heterogeneity of the data, the lack of structure, and the unpredictable nature of its generation. Frankly, I have no doubt that the NSA is not monitoring phone chatter on a mass scale, probably not because they can't, but because if they did there would be no way in hell to parse, store, process and evaluate the data generated.

We (the scientific/big data community) can barely get recommendation engines working well - engines which have one set of data (what you watched) and do one other thing (suggest what else you might want to watch). Unless the NSA is decades ahead in a number of fields (like data warehousing, statistical analysis of massive datasets, machine learning) how are they getting useful information in a systematic way, considering the pressure from the data-firehouse involved?

My guess is they're probably not - instead the data are collected, and then used in conjunction with traditional approaches. e.g. little johnny buys some fertilizer and one way plane ticket - so who's he been talking to, what's he been saying, etc.

Honestly, how the NSA is using/dealing with/storing/accessing these data is actually an incredibly interesting question, from an academic/systems perspective.

If they're looking for patterns similar to those of historical terrorists, then their false positive rate is likely reasonably low considering the scale of their data set.

If they're looking for patterns or traits of hypothetical terrorist behavior, that's another story.

Systems-based trading in the financial sector comes to mind. Constructing a trading system that performs well when tested against historical data is easy. Constructing a system that performs well on future data isn't.

The solution to the latter usually involves using more generalized indicators when building the system to avoid the pitfall of curve-fitting your system to the data.

In this case though, it might just mean more false positives to sift through.

So either they know it's ineffective and do it anyway, because they can. More money, more power, more influence.

Or it's not being used to generate leads, but as a way to look up retrospectively what people have done online once they become of interest from tip-offs and traditional investigations.

Like say for example, you catch a terrorist but he won't tell you anything and you suspect they were not acting alone.

So, maybe you interview the guy's brother who insists he knows nothing , hasn't seen his brother for 5 years and loves America.

So you check the brother out with PRISM and find that: He had an IM conversation with someone in 2002 and he spoke about how happy he was that 9/11 happened.

Someone had taken and uploaded a photo to a social network of the brothers at the same place 2 years ago.

You decide that putting covert surveillance on the brother might not be a waste of resources.