XKeyscore: NSA program collects 'nearly everything a user does on the internet'

Again, I'll chime in as the resident apologist. The people working at Fort Meade are not evil. They truly believe they're doing a great service to the nation. They may be wrong, and they've certainly thrown privacy out the window. But they are following an ideal: national security.

Post 9/11, the nation went on a war footing. We reacted the way we did to the Nazis and the Soviets. And in their search for an existential threat, the intelligence community seized on nuclear terrorism. These analysts live in constant fear of the day they miss a piece of information and New York, Washington, or London is enveloped in a mushroom cloud.

The best explanations for this type of reasoning that I have heard came from an unlikely source, my grandfather. He's a former FBI agent and WWII Navy veteran. In war time, we threw all sorts of civil, economic, and political liberties out the window to defend ourselves. When I asked him how this was allowed to happen, he said simply, "When you're facing an enemy that wants to cross over the hill into the valley where you, your family, and everyone you've ever known or loved lives, you'll do anything to protect them."

Our grandparents grew up with the threat of the Nazis. Our parents faced the prospect of annihilation by the Soviets. We have had the luxury of coming of age in a time where there is no credible threat to our very national and physical existence.

As a result, it's difficult for us to understand the mindset of someone that spends all day, every day, thinking of the most horrible ways we could be attacked, and then trying to devise countermeasures. It's almost inevitable their perspective on the balance between security and privacy is altered.

I'm not saying this reasoning is morally correct or justifiable, especially when applied to the current surveillance programs, but simply that it is understandable.

The key danger is that these efforts are qualitatively distinct from those in previous generations. The difference between extraordinary measures now and then is twofold.

First, our capacity to surveil the citizenry has exploded over the past two decades, and our legal framework is still grappling with that change. The courts are having trouble understanding that a change in scale can be a change in kind.

For example, it's one thing to have the occasional surveillance flight to search for drug operations. It's quite another to have aerostats and quadrotors watching every inch of a city all the time. But the legal rational that there is no right to privacy in public spaces allows both.

Similarly, it's one thing to say the records generated by my water company are business records not subject to the Fourth Amendment, but it's quite another to use that rationale to justify monitoring the location of my cell phone simply because my cellular provider maintains the records.

Second, wars have a point where they end, and the extraordinary measures are supposed to be reversed. That's why the "war on terror" and the "war on drugs" are so dangerous to civil liberties. They essentially extend the extraordinary measures during wartime to police problems that have no logical end.

I agree that we've gone too far as a nation. The fact that these queries don't require FISA orders flat out shocked me, even as a careful observer of these issues. But let's not demonize the individuals. After all, they're only doing what the people demanded after we were attacked. This is a democracy, and immediately after 9/11 such measures were resoundingly approved by the public and our representatives, beginning with the PATRIOT Act.

None of that changes the current reality however. We must slowly learn the lesson the British did when dealing with terrorism. If you treat it as an ordinary police matter, something that will always be present, you deprive it of its power to shock, from which it derives its effectiveness.

The fact is that the war on terror must now end. It's time for a return to normalcy.

As to the how they are taking feeds directly off major internet routers (the vast majority of traffic will go through a major router at some point particularly if it is international though it's quite possible for a packet sent from one side of your country to another to go international as well).

So yes they do have email content capabilities (if you look at the actual slides they also have a sophisticated filtering system, they can do stuff like "show me emails from iran with word documents attached containing IAEO").

This system is absolutely terrifying, it genuinely is the work of a dystopian sci-fi author from 30 years ago.

----

If you want to get right down in the trenches email is SMTP and POP over TCP/IP (normally), email is fundamentally a human readable text protocol which makes it trivially easy to parse (this was kind of the intention after all) so once they have the captured stream reconstructing the mail is not much harder (if any) than writing a mail client.

You can see an example of SMTP if you open a console/shell and type "telnet smtp.gmail.com 25" and then when it has logged in type HELO the response is just plain text.