Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook-owned sites were down

facebook.com

Post reply on HN
Print view View original

711–720 of 1001 posts

Re: Facebook-owned sites were down

#711

Post by Scoundreller »

gbil wrote: post #383
pmlnr wrote: post #314

Earlier quoted context omitted.

> the people with physical access is separate from the people with knowledge of [...] Welcome to the brave new world of troubleshooting. This will seriously bite us one day.

this is not new, this is everyday life with helping hands, on duty engineers, l2-l3 levels telling people with physical access which commands to run etc. etc. etc.

Then you have security issues like this where someone impersonates a client with helping hands and drains your exchanges hot wallet:

https://www.huffpost.com/archive/ca/entry/canadian-bitcoins-...

Re: Facebook-owned sites were down

#712

Post by cududa »

okwubodu wrote: post #682
kiernanmcgowan wrote: post #643

Earlier quoted context omitted.

My suspicion is that since a lot of internal comms runs through the FB domain and since everyone is still WFH, then its probably a massive issue just to get people talking to each other to solve the problem.

I don’t know how true it is but a few reports claim employees can’t get into the building with their badges.

I remember my first time having a meeting at Facebook and observing none of the doors had keyholes and thinking "hope their badge system never goes down"

Re: Facebook-owned sites were down

#713

Post by user3939382 »

MrYellowP wrote: post #642

I keep trying to submit to HN but I keep getting an error. What's wrong with the internet? FaceBook is down. My friend from Slovenia is having trouble with discord. It eats his messages. I can't load photos from my friend in telegram and the messages take a relatively long time - multiple seconds! - to get received. TrackMania players have talked about having input lag. ycombinator is really slow and reports an error…

Big ISP outages in NYC right now

Re: Facebook-owned sites were down

#714

Post by _fizz_buzz_ »

emmap21 wrote: post #683

Not only facebook, but also Google, Zoom, Telegram, Youtube and many more internet service/ product/ providers from 8:00 AM today. This is more like internet outage.

Youtube and google are definitely working for me without any problems (haven't tested telegram or zoom).

Re: Facebook-owned sites were down

#715

Post by Animats »

MrYellowP wrote: post #644

I keep trying to submit to HN but I keep getting an error. What's wrong with the internet? FaceBook is down. My friend from Slovenia is having trouble with discord. It eats his messages. I can't load photos from my friend in telegram and the messages take a relatively long time - multiple seconds! - to get received. TrackMania players have talked about having input lag. ycombinator is really slow and reports an error…

What's wrong with the internet?

With Facebook down, some large DNS servers seem to be struggling with the extra load of failing requests to look up "facebook.com". Cloudflare reports overload with their DNS server at 1.1.1.1, although that's working for me.

Billions of things worldwide are trying to connect to Facebook. The lookup which normally returns the IP address for facebook.com on the first try now requires trying a.ns.facebook.com, b.ns.facebook.com, etc. several times each before giving up. Probably several times a minute for everyone who has a Facebook app in their phone turned on. That may be using a big fraction of world DNS resources.

Vodaphone Ireland seems to be struggling with a DNS overload right now, per the Irish Independent. Also, their status page can't find "Dublin" as a city.

Re: Facebook-owned sites were down

#716

Post by wtf-is-ur-prblm »

yholio wrote: post #648
guidopallemans wrote: post #476

Earlier quoted context omitted.

He just deleted all his updates. user: https://old.reddit.com/user/ramenporn some messages: * This is a global outage for all FB-related services/infra (source: I'm currently on the recovery/investigation team). * Will try to provide any important/interesting bits as I see them. There is a ton of stuff flying around right now and like 7 separate discussion channels and video calls. * Update 1440 UTC: \ As many of you…

Essentially, they locked themselves out with an uninspired command line at the exact moment the datacenter was being hijacked by ape-people. Yup, corporate comms won't love these status updates.

Sorry, are you referring to data center technicians as “ape people”?

Re: Facebook-owned sites were down

#717

Post by reilly3000 »

pytlicek wrote: post #697

Who else sees their deleted messages on WhatsApp that shouldn't be there? https://twitter.com/Pytlicek/status/1445072626729242637

That is wild and definitely newsworthy. Capture as many screenshots and data as you can.

FWIW it seems possible that the messages remain cached locally on your device but deleted from their servers, and with their outage they aren’t being updated to delete?

Re: Facebook-owned sites were down

#719

Post by lucidbee »

The timing of this is so rich in irony I can't help but wonder if there is an element of internal sabotage. How many FB employees hate FB right now? The latest expose of FB is both effective and truly awful. I can't imagine feeling good about a FB job. And it's gotten worse! Now they look like they can't even keep their websites up.
Post reply on HN