Facebook-owned sites were down • Hacker News Board

Re: Facebook-owned sites were down

#191

Post by bborud » Mon, Oct 04, 2021, 4:20 PM UTC

finolex1 wrote: post #57

RedShift1 wrote: post #7

Nothing of value was lost

I get that this in jest, but a lot of people rely on Whatsapp and FB Messenger for messaging.

Maybe they shouldn't.

Re: Facebook-owned sites were down

#192

Post by jamespwilliams » Mon, Oct 04, 2021, 4:21 PM UTC

Is this caused by missing glue records? I can’t resolve any of FB’s nameservers. Anyone know how that could happen?

Re: Facebook-owned sites were down

#193

Post by zekrioca » Mon, Oct 04, 2021, 4:21 PM UTC

yawnxyz wrote: post #113

Funny enough, I went to https://www.isitdownrightnow.com/ to check if Facebook is down, and isitdownrightnow is down itself... probably from the massive number requests coming to check if Facebook is down

Which in turn, reminds of this paper [1] (from someone who previously worked at Facebook).

TLDR; Metastable failures occur in open systems with an uncontrolled source of load where a trigger causes the system to enter a bad state that persists even when the trigger is removed.

[1] Metastable Failures in Distributed Systems - https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s...

Re: Facebook-owned sites were down

#194

Post by rocho » Mon, Oct 04, 2021, 4:21 PM UTC

elboru wrote: post #169

Is it just me or HN also feels kinda laggy?

I can confirm, HN, GitHub and Slack are very slow for me as well. Google is very fast, on the other hand.

Re: Facebook-owned sites were down

#195

Post by hkai » Mon, Oct 04, 2021, 4:21 PM UTC

drcongo wrote: post #66

Let's hope it's permanent.

Why?

Re: Facebook-owned sites were down

#196

Post by vodkapump » Mon, Oct 04, 2021, 4:22 PM UTC

yawnxyz wrote: post #168

I thought Facebook, Instagram and WhatsApp ran on different infrastructure (and they've been trying for a while to align everything)? How could they all go down at the same time, if they have different teams of engineers running each product separately? Could anyone with some background (or person familiar with the matter) explain how their system's set up?

Seems unrelated to their infrastructure, the DNS records for facebook.com, instagram.com, whatsapp.com and all derivative domains are wiped clean it seems

edit: though saying that, they do run their own registrar... Might've fucked something up over there.

Re: Facebook-owned sites were down

#197

Post by rocho » Mon, Oct 04, 2021, 4:22 PM UTC

elboru wrote: post #169

Is it just me or HN also feels kinda laggy?

I can confirm, HN, GitHub and Slack are very slow for me as well. Google is very fast, on the other hand.

EDIT: actually HN failed to post this comment the first time I posted it!

Re: Facebook-owned sites were down

#198

Post by Spare_account » Mon, Oct 04, 2021, 4:22 PM UTC

WillPostForFood wrote: post #166

rocho wrote: post #36

For Facebook and WhatsApp it looks like a DNS issue, name resolution fails with SERVFAIL: $ dig facebook.com ; > DiG 9.16.21 > facebook.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER

I'm seeing similar DNS errors for many non-Facebook sites.

Do you have some examples?

Re: Facebook-owned sites were down

#199

Post by eeegnu » Mon, Oct 04, 2021, 4:22 PM UTC

elboru wrote: post #169

Is it just me or HN also feels kinda laggy?

Probably people flooding in to see if anyone knows why things are down. Even Google speed test was down, presumably from too many people testing if it's their internet that's at issue.

Re: Facebook-owned sites were down

#200

Post by neom » Mon, Oct 04, 2021, 4:22 PM UTC

elboru wrote: post #169

Is it just me or HN also feels kinda laggy?

HN lagging, BBC was also very laggy about 30 minutes ago, and 35 minutes ago our whole company got booted out of their various hangouts simultaneously apart from the people in the states.