I'm one of the hard-headed privacy freaks usually sharpening my pitchfork when there is an outrage against civil liberties. I'm
that guy.
I once had a job that involved investigations of criminal activity (not law enforcement or government related, just a company protecting its own users and employees).
In this case, I had identified, with certainty, one individual that was engaging in significant fraud. He appeared to have several accounts, and it was appearing highly likely that he had a few accomplices.
During the investigation, I was fully willing to violate everyone's privacy to find everyone in the fraud network. This included data that was already submitted voluntarily, private communications, as well as embedding tracking objects and invisible flash objects to retrieve IP addresses of users surfing behind proxies (this used to be an effective way to unmask users). I didn't have a second thought about it. Why would I? I didn't care what the legitimate users were doing, wasn't going to stalk them, wasn't going to pay any attention to their personal affairs. But, to weed out this problem effectively, I needed to sweep everything. I'm trustworthy, just doing my job, and I certainly trust myself enough to disregard or ignore information that wasn't pertinent.
After being entrenched in the investigation, I had a fairly exhaustive list of the bad actors. Initially this was just basic hard data, (such as correlating IP addresses), but then there was kind of a "sixth sense" that I also started relying on, where I couldn't articulate the signal, but some behavioral cues just felt like they were related. You know, "gut instinct". So I ended up digging into those accounts, and confirmation bias took over. I did find many more bad actors, but I was thoroughly convinced that a few cases were also related, which ended up being suspended, and it turned out that they were actually unrelated and legitimate. That's when I started to reflect a bit.
I didn't go through with the most blatant of the proposed violations, although at the time I was willing to initially. I now realized how egregious that was, and noticed how easily I fell into that mindset. If asked, I think the words "If you've got nothing to hide, you've got nothing to fear" could have naturally rolled off my tongue (though, this certainly would have alerted me to the errors of my thought process).
So I concluded a few things:
- Most of the time, these blatant, sweeping violations, are most likely not malicious and probably do have good intentions. I very much understand what frame of mind most of those people are in. It's not an opaque three letter agency, it's made up of regular individuals with tunnel vision on their legitimate objectives (stopping crime).
- When you look at criminals day in and day out, and are on a mission, everybody starts to look like a criminal.
- The "working backwards" approach - finding signatures of bad activity, and applying it to other data, then "confirming" the new matches, is a well-understood statistical fallacy, aptly named, the prosecutor's fallacy[1]. If you spot it in court, your defense attorney can try and point it out to the jury - and good luck explaining it to your "peers" who probably play the same lotto numbers because theirs is "due eventually". But let's face it - your life is already ruined by then. You're on all the watch lists, your vehicles are bugged, you've got huge legal bills and no job, and maybe if you're extremely unlucky, you're even in Guantanamo. Everything prior had little or no judicial oversight, no way to defend yourself, and is from a system that is invariably full of investigators who are not self-aware enough to always catch themselves doing this, especially when the cost of missing an actual threat is extremely high.
And for bonus points:
The interface that a coworker created to do some of the data mining (let's call it the "lawful intercept interface") had an SQL injection bug in the logic that parsed login history. It wouldn't have been difficult to discover and exploit without even knowing this interface existed, due to the error a user would see on login if they had certain bad characters in the affected field. I found it roughly a year later and reported it to the CTO in a message from his own account, after using the bug to take his auth cookie out of the DB (we were friends, so I knew he would be a good sport).
tl;dr It's mostly good intentioned individuals with tunnel vision, who are very misguided, and who don't understand the side effects and costs of what they propose.
[1] http://en.wikipedia.org/wiki/Prosecutor%27s_fallacy