Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding how Facebook disappeared from the internet

blog.cloudflare.com

Post reply on HN
Print view View original

231–240 of 280 posts

Re: Understanding how Facebook disappeared from the internet

#231

Post by Philip-J-Fry »

codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

I'd just say that you had an address for Facebook in your address book. The page somehow vanished and you don't know their address any more. So you start phoning other people and knocking on their door to try and find what their address is. Everyone else is doing this and no one knows what their address is. So you've got millions of people phoning each other and knocking on doors.

Facebook being down was already an issue, but everyone phoning and knocking on doors was causing disruption to everyone else.

Re: Understanding how Facebook disappeared from the internet

#232

Post by cjrp »

acomjean wrote: post #191
gorgoiler wrote: post #51

Earlier quoted context omitted.

At a guess: reconnecting traffic at the billions-of-people scale has the potential for finding all sorts of weird behaviours. For all we know, it has been connected and disconnected 10x already during the outage, with each reconnect overwhelming some new, deeper level of the system each time. Reconnect and the GLBs fall apart under load as the entire world’s cadre of recursive resolvers hit you. Fix that. Reconnect a…

Reminds me of when I was working at a startup we DOSed ourselves. We had devices that monitored power minute by minute. A load balancer was mis configured and we went down for a some hours. We came back up and the devices all saw that and flooded us with all the data they’d been storing since we were down.. down again… bring it up and down again…. we needed a better fix. I was up till 2 am with the other developer co…

God bless exponential backoff!

Re: Understanding how Facebook disappeared from the internet

#234

Post by teh_klev »

edf13 wrote: post #233

Didn't we used to have long TTLs on DNS hostnames to help with this very reason?

Once upon a time this would have been the norm. If we were performing any DNS related maintenance we'd drop the TTL to 5 mins a day or so before the maintenance window was due to start. Once happy we'd not broken anything then we'd bump back to 21600 or whatever. But I guess the move fast/break all the things crowd no longer have any patience for this kind of thing.

Re: Understanding how Facebook disappeared from the internet

#235

Post by dgb23 »

nroets wrote: post #224
PaulRobinson wrote: post #208

It's been 20+ years since I had reason to know the internals of BGP, but I still carry a slight nervousness whenever I think about it. In many ways, it's beautiful in that it allows Autonomous Systems to come together and independently make the Internet. However, I always felt it was brittle in that it relied every AS to have edge routers aware of the entire AS routing map, and so when those routers went down hard, w…

So much of the internet is about translating addresses from one format to another: Human readable to IP: DNS Private IP to Public IP: NAT MAC to Private IP: ARP Public IP to AS: BGP With a block chain DNS it may be possible to replace all of them with a single protocol, but it may well turn out to be an academic exercise.

These are abstraction barriers that isolate locations, layers and protocols from each other. I don't understand this as translation but rather as a separation of concerns.

Re: Understanding how Facebook disappeared from the internet

#236

Post by thorin »

goatlover wrote: post #195
nimbius wrote: post #64

Earlier quoted context omitted.

Unpopular opinion but I think a talent exodus and turnover are likely at fault here as well. The people who stood up this juggernaut are no longer here, and Facebooks ability to consistently attract talent that is competent enough to maintain such a formidable beast is hampered by repetitive revelations that it operates at the net-loss of humanity as a whole. Facebooks no longer an innovator, just a mining operation…

Isn’t Facebook huge in some Asian countries? Supposedly 3.5 billion people use one or more of its services. WhatsApp, Instagram and Oculus certainly aren’t just used by the elderly.

Anecdotally around 90% of the people I know in the UK (40s age group ) have Facebook and use it for something (even if it's just for checking restaurants/gigs etc), almost everyone uses Messenger or Whatsapp. Also for those saying they use Facetime, stats suggest that half phones in the UK are Android. FB is relevant, even if you want to be, maybe in the same ways Windows/Microsoft is relevant for a whole load of people.

Re: Understanding how Facebook disappeared from the internet

#237

Post by dgb23 »

cortesoft wrote: post #155
rjvir wrote: post #79

Earlier quoted context omitted.

It's merely an informative blog post on a topic many people are interested in. I see nothing wrong with Cloudflare just explaining what happened, even though they had nothing to do with it.

It is little stuff like this that makes it come off a bit self-aggrandizing: > We keep track of all the BGP updates and announcements we see in our global network. At our scale, the data we collect gives us a view of how the Internet is connected and where the traffic is meant to flow from and to everywhere on the planet. > Fortunately, 1.1.1.1 was built to be Free, Private, Fast (as the independent DNS monitor DNSPe…

It's a proud and confident type of marketing. A bit on the nose.

I very much prefer that over the almost patronizing, overly friendly tone some others have, or the stripped of any personality style that most have.

Re: Understanding how Facebook disappeared from the internet

#238

Post by BFG-One »

xtat wrote: post #102
Ansil849 wrote: post #72

Is anyone else kinda put off by how Cloudflare keeps interjecting themselves into this situation? First with the Twitter posts and now this blogpost. They're a passive observer, not involved in this event, and yet they keep putting themselves in the middle of it. Cloudflare staff are not Facebook staff, they do not know how things actually went down to trigger the events that transpired. They are essentially doing gl…

Cloudflare are also good at disappearing people from the Internet

I don't trust any analysis from CloudFlare. These are the same people preventing people with vpns or tor browsers from reaching sites while screaming and yelling OMG DDoS!! then blaming the customer of CloudFlare for not knowing how to configure it.

Re: Understanding how Facebook disappeared from the internet

#239

Post by BFG-One »

codedeadlock wrote: post #206

How can you explain yesterday's outage (Facebook, Instagram, WhatsApp) to your parents? You are feeling hungry and went to food court. The food court (open area) has a lot of options. You sit down in front of Domino's (Facebook), since you want to eat garlic bread. Now, you can't order from the counter directly. The waiter will come to your seat and ask for the order. You ordered garlic bread from the waiter, but the…

cool, but were did mark zuckerberg (dominos chef) go? he felt dissed so he ran and hid?

Re: Understanding how Facebook disappeared from the internet

#240

Post by BFG-One »

5560675260 wrote: post #218
codedeadlock wrote: post #210

Earlier quoted context omitted.

Thanks for the feedback. I will simplify it further!

There is no need to make it any more complex than "facebook, the company, messed up, now their properties are broken". An overly elaborated analogy just makes you sound condescending.

don't think the parents (target audience) would think he sounds condescending.
Post reply on HN