Live data from Hacker News

Update about the October 4th outage

engineering.fb.com

81–90 of 239 posts

Re: Update about the October 4th outage

#81

Gotta love how painfully vague this is. Sounds like a PR piece for investors, not an engineering blog piece.

I think you need to re-adjust your expectations, it's not reasonable to have a fully fleshed out RCA blog post available within hours of incident resolution. Most other cloud providers take a few days for theirs.

I mean, not an RCA per se, but info more akin to cloudflare's blog post would be v welcome IMHO: https://blog.cloudflare.com/october-2021-facebook-outage/

Re: Update about the October 4th outage

#83
post #41

Earlier quoted context omitted.

I didn’t see any disinformation, just initial reports that it was DNS which were later explained to be caused by BGP.

- It was government intervention - Facebook was hacked - They did it on purpose to bury the whistleblower story - No one could access Facebook offices - They had to cut open servers with angle grinders - Disgruntled employees changed DNS records - Lots of made up numbers for how much money Facebook/the rest of the economy was losing (or gaining) They probably rushed out this blog post just to dispel some of these rum…

You do know the meaning or jokes and/or ironic comments, right?

Re: Update about the October 4th outage

#84

> We also have no evidence that user data was compromised as a result of this downtime. I am not sure why they had to mention this specifically. This makes it sound like an external attack.

It doesn't make it sound like an attack, it's standard boilerplate to dispel any worries. It's natural for anyone to wonder downtime -> data loss? , so it's natural to reassure people that it wasn't the case.

Re: Update about the October 4th outage

#85
post #71
post #21

It was quite ironic that while every Facebook property was offline there was an immense amount of misinformation about the incident perpetuated across the internet (including right here on HN) which everyone just believed as fact.

I work in a different social media company that has had some visible outages. Its always hilarious to see how wrong people are with their confident speculation. It's a good reminder that people online are often full of shit.

I work in video games.

It amazing how wrong people can be and how confident they are about being right.

Even sometimes fighting _me_ about things _I_ designed and built.

It’s quite sobering; taught me not to believe all the speculation I read.

Re: Update about the October 4th outage

#86
The badge story only shows how people are looking for "efficiency" where it doesn't matter, with predictable results.

The badge system should be local to the building. There are few actual reasons (sure, besides "efficiency") of why badge control should be centralized. Even less reasons for it to be a subdomain of fb. Another option would be to keep the system but make it failsafe (but it seems the newer generation doesn't know what that means). If the network goes down keep it at the last config. Badge validation should be offline first and added/removed ones should be broadcast periodically.

This is the same issue with smartlocks times the number of employees. Do you really want to add another point of failure between yourself and your home?

Re: Update about the October 4th outage

#87
It just occurred to me to wonder if Facebook has a Twitter account and if they used it to update people about the outage. It turns out they do, and they did, which makes sense. Boy, it must have been galling to have to use a competing communication network to tell people that your network is down.

It looks like Zuckerberg doesn't have a personal Twitter though, nor does Jack Dorsey have a public Facebook page (or they're set not to show up in search).

Re: Update about the October 4th outage

#88
post #85
post #71

Earlier quoted context omitted.

I work in a different social media company that has had some visible outages. Its always hilarious to see how wrong people are with their confident speculation. It's a good reminder that people online are often full of shit.

I work in video games. It amazing how wrong people can be and how confident they are about being right. Even sometimes fighting _me_ about things _I_ designed and built. It’s quite sobering; taught me not to believe all the speculation I read.

[deleted]

Re: Update about the October 4th outage

#90
post #21

It was quite ironic that while every Facebook property was offline there was an immense amount of misinformation about the incident perpetuated across the internet (including right here on HN) which everyone just believed as fact.

The problem is not misinformation per se but the largest social media data processor running algorithms boosting such information for profit.
Post reply on HN