Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding how Facebook disappeared from the internet

blog.cloudflare.com

Post reply on HN
Print view View original

211–220 of 280 posts

Re: Understanding how Facebook disappeared from the internet

#211

Post by ccakes »

codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

You need to call Facebook to find out what your friends have been up to but their number has temporarily disappeared from the phone book.

Re: Understanding how Facebook disappeared from the internet

#212

Post by p4bl0 »

Hikikomori wrote: post #66
adamcharnock wrote: post #27

Earlier quoted context omitted.

I’m pretty new to BGP, but I’d imagine that cutting off access to an AS is fast because all it takes is for the neighbouring routers update their routes. At which point any traffic that makes it that far is simply dropped. Whereas to make an announcement, the entire internet (or at least all routers between the AS and the user) need to pickup the new announcement. (Note: I still need to read the article)

Once they start adversiting again it should only take a few minutes at most for most ISPs to get it.

But then it is the DNS that had to propagate from the now accessible authoritative servers.

Re: Understanding how Facebook disappeared from the internet

#213

Post by numpad0 »

goatlover wrote: post #195
nimbius wrote: post #64

Earlier quoted context omitted.

Unpopular opinion but I think a talent exodus and turnover are likely at fault here as well. The people who stood up this juggernaut are no longer here, and Facebooks ability to consistently attract talent that is competent enough to maintain such a formidable beast is hampered by repetitive revelations that it operates at the net-loss of humanity as a whole. Facebooks no longer an innovator, just a mining operation…

Isn’t Facebook huge in some Asian countries? Supposedly 3.5 billion people use one or more of its services. WhatsApp, Instagram and Oculus certainly aren’t just used by the elderly.

Depends on which specific country you mean by Asia, as nations are kind of culturally segregated by sea and by heritage in Asia - I mean, Instagram is but Facebook itself is smol in Japan[1] and Twitter is bigger unlike anywhere[2] (taken from page 110 and page 170 of "The Digital 2021 Global Overview Report" from we are social/Hootsuite[3]).

1: https://image.slidesharecdn.com/datareportal20210308gd001dig...

2: https://image.slidesharecdn.com/datareportal20210308gd001dig...

3: https://wearesocial.com/blog/2021/01/digital-2021-the-latest...

Re: Understanding how Facebook disappeared from the internet

#215

Post by philposting »

codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

I explained it as Facebook the city still existing, but they'd taken down the signposts.

Re: Understanding how Facebook disappeared from the internet

#216

Post by phtrivier »

ccakes wrote: post #211
codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

You need to call Facebook to find out what your friends have been up to but their number has temporarily disappeared from the phone book.

That would have been accurate for a DNS outage ; but with my layman understanding of BGP, I would say the analogy would be something between "...but their phone line is broken" and "...but they disappeared from the phone book because they don't have a phone line any more" .

Is that right ?

Re: Understanding how Facebook disappeared from the internet

#217

Post by heavenlyblue »

acomjean wrote: post #191
gorgoiler wrote: post #51

Earlier quoted context omitted.

At a guess: reconnecting traffic at the billions-of-people scale has the potential for finding all sorts of weird behaviours. For all we know, it has been connected and disconnected 10x already during the outage, with each reconnect overwhelming some new, deeper level of the system each time. Reconnect and the GLBs fall apart under load as the entire world’s cadre of recursive resolvers hit you. Fix that. Reconnect a…

Reminds me of when I was working at a startup we DOSed ourselves. We had devices that monitored power minute by minute. A load balancer was mis configured and we went down for a some hours. We came back up and the devices all saw that and flooded us with all the data they’d been storing since we were down.. down again… bring it up and down again…. we needed a better fix. I was up till 2 am with the other developer co…

Why couldn’t you just statistically drop traffic?

Re: Understanding how Facebook disappeared from the internet

#218

Post by 5560675260 »

codedeadlock wrote: post #210
miked85 wrote: post #209

Earlier quoted context omitted.

I think that this may be more confusing than the actual explanation.

Thanks for the feedback. I will simplify it further!

There is no need to make it any more complex than "facebook, the company, messed up, now their properties are broken". An overly elaborated analogy just makes you sound condescending.

Re: Understanding how Facebook disappeared from the internet

#219

Post by samhw »

5560675260 wrote: post #218
codedeadlock wrote: post #210

Earlier quoted context omitted.

Thanks for the feedback. I will simplify it further!

There is no need to make it any more complex than "facebook, the company, messed up, now their properties are broken". An overly elaborated analogy just makes you sound condescending.

He's trying to explain the knock-on effect on other websites. I don't see how this is at all condescending.

Re: Understanding how Facebook disappeared from the internet

#220

Post by rvnx »

codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

Another way to see the situation:

Internet is just a bunch of computers interconnected via tons of cables (hence the name; "inter-networked computers").

To be reachable, every equipment and computer constantly need to tell the others about their existence (to publicly announce on which network cable they can be reached at).

Facebook engineers wanted to optimise that system but accidentally broke it during the update.

As a consequence, after a few minutes, other computers didn't know on which cables they can reach Facebook.

Facebook had to call the technicians sitting in the datacenter to cancel the last change that was done (because the Facebook engineers couldn't themselves connect from the office) and everything was fine again.

Post reply on HN