Facebook and the Normalization of Deviance

Facebook's censorship (which includes algorithmic timelines, prioritizing some things and censoring others) is an existential danger to the wellbeing of our society. Any organization that can decide what links and text can be shared from neighbor to neighbor or from friend to friend is in a position of massive power.

They even prohibit links to certain tweets in DMs.

Re: Facebook and the Normalization of Deviance

#8

Post by Ozzie_osman » Mon, May 03, 2021, 2:55 AM UTC

This is the "move fast and break things", "grow at all cost", "ask for forgiveness instead of beg for permission" culture. Facebook is the most obvious and visible example of this, but I fear it's permeated far beyond that in our industry.

Re: Facebook and the Normalization of Deviance

#9

Post by jchw » Mon, May 03, 2021, 2:56 AM UTC

Note for headline-only readers: the “normalization of deviance” here refers to organizational issues, referencing the term as it was coined in regards to the events leading up to the Challenger incident. It is not, as far as I can tell, intended to be a stigmatizing remark about behavior considered “deviant.” It’s probably worth noting this since it’s easy to interpret it that way from the headline.

Re: Facebook and the Normalization of Deviance

#10

Post by staticassertion » Mon, May 03, 2021, 3:00 AM UTC

I was just thinking the other day about what it must be like to work for Facebook on their security team (I'm in security myself). On an infosec team I have an idea of who the attackers are - there may be multiple, ranging from disgruntled employees to nation states. I think about their incentives, capabilities, and how to reduce risk.

At Facebook... I'm working for the attacker. You're protecting users from some set of other attackers, of course, but like, if the idea here is "reduce harm to the user", you're working for one of those attackers.

Facebook is essentially normalizing a breach. It's not really particularly different from some threat actor gaining access to a database, except that the users "consent" (if such a thing is possible - as if users understand the implications of their consent, not to mention the tracking that happens without consent).

As the post notes, and as Facebook states, " Longer term, though, we expect more scraping incidents and think it’s important to . . . normalize the fact that this activity happens regularly."

Of course, this makes it clear that protecting users isn't the goal, protecting the company's interests is the goal. It feels like engineers and others may be in need of a hippocratic oath - a commitment that makes it clear that, while you may be employed by some entity, you must do no harm. You have ethical responsibilities that extend beyond the scope of your employment to that entity.

> Neither will I administer a poison to anybody when asked to do so, nor will I suggest such a course.