Live data from Hacker News

Facebook-owned sites were down

facebook.com

221–230 of 1001 posts

Re: Facebook-owned sites were down

#221
post #131

Funny enough, I went to https://www.isitdownrightnow.com/ to check if Facebook is down, and isitdownrightnow is down itself... probably from the massive number requests coming to check if Facebook is down

https://downdetector.com/ seems to be working for me at least.

It's amusing that the top 3 trending reports are the FB sites that are down, and then the mobile carriers themselves, presumably because when FB doesn't load they assume it's their mobile network's fault. People really do think FB is the internet

Re: Facebook-owned sites were down

#224
I had problems with my internet connection and loaded my ISPs site. Strangely, my bill was paid. Even stranger, some sites load while others do not.

Then it hit me: I am so dependent on Facebook owned properties (Whatsapp, Facebook, insta) that a Facebook failure looks to me like an internet failure.

Re: Facebook-owned sites were down

#225
post #199
post #169

Is it just me or HN also feels kinda laggy?

Probably people flooding in to see if anyone knows why things are down. Even Google speed test was down, presumably from too many people testing if it's their internet that's at issue.

https://www.speedtest.net is down too

Re: Facebook-owned sites were down

#226

Just wondering - would the engineer who made the mistake be fired?

"Asking for a friend."

I kid. If it were to come down to a single person, that's really a failure of the whole organization system and not of the individual.

This apocryphal [1] punchline to the Jack Welch story also sums up how most orgs deal with this sort of thing:

"I just spent a million dollars on your education - why would I fire you now?"

[1]: http://www.nickmilton.com/2016/03/jack-welch-on-learning-fro...

Re: Facebook-owned sites were down

#230

Seems to be DNS related. None of the listed facebook nameservers are resolvable or reachable: a.ns.facebook.com b.ns.facebook.com c.ns.facebook.com d.ns.facebook.com

Looks like the routing is goofed. Loops over and over - DDoS attacking themselves. mtr -r -c10 -w -b a.ns.facebook.com Start: 2021-10-04T10:02:50-0600 Loss% Snt Last Avg Best Wrst StDev ... 4.|-- ae-2-rur101.cosprings.co.denver.comcast.net (162.151.51.125) 0.0% 10 12.6 11.9 9.6 19.0 2.9 5.|-- 24.124.155.233 0.0% 10 9.3 10.2 9.1 12.4 1.1 6.|-- 96.216.22.45 0.0% 10 12.0 14.0 11.6 31.3 6.1 7.|-- be-36041-cs04.1601milehi…

Same issue over at b.ns.facebook.com Looping routing creating self-inflicted DDoS

mtr -r -c10 -n b.ns.facebook.com

Start: 2021-10-04T10:28:03-0600 Loss% Snt Last Avg Best Wrst StDev

  1.|-- 192.168.1.1                0.0%    10    0.2   0.2   0.2   0.3   0.0 

  2.|-- 96.120.12.229              0.0%    10   10.2  10.8   8.8  15.7   1.9 

  3.|-- 96.110.149.185             0.0%    10   17.7  13.6   9.8  32.3   7.0 

  4.|-- 162.151.51.125             0.0%    10   10.9  12.2   9.6  15.3   1.9 

  5.|-- 24.124.155.233             0.0%    10   13.0  10.4   9.4  13.0   1.2 

  6.|-- 96.216.22.45               0.0%    10   16.5  16.7  11.2  29.1   6.4 

  7.|-- 96.110.43.241              0.0%    10   17.4  13.6  11.9  17.4   1.6 

  8.|-- 96.110.38.114              0.0%    10   12.5  12.8  12.0  14.0   0.6 

  9.|-- 173.167.59.170             0.0%    10   36.1  19.3  11.6  36.1   9.7 

 10.|-- 129.134.40.76              0.0%    10   13.1  12.3  11.3  13.1   0.6 

 11.|-- 129.134.34.72              0.0%    10   15.3  15.7  13.5  21.3   2.5 

 12.|-- 129.134.102.85             0.0%    10   39.0  39.2  38.0  40.8   1.0 

 13.|-- 31.13.25.13                0.0%    10   30.5  29.8  28.5  31.0   0.9 

 14.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0 

 15.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0 

 16.|-- 31.13.25.13               90.0%    10   30.2  30.2  30.2  30.2   0.0
On a side note... why is it so freaking hard to format line breaks in HN?
Post reply on HN