Earlier quoted context omitted.
That’s some really good thoughts on DR planning. I have never thought DR to be to such an extent. How many companies really plan for an event where their entire infrastructure goes offline and their entire team gets killed? Does even companies like Google plan for this kind of event?
Yes, Google plans extensively and runs regular drills. It's hearsay, but I was once told that achieving "black start" capability was a program that took many years and about a billion dollars. But they (probably) have it now.
Slack is down
461–470 of 840 posts
Re: Slack is down
#462Earlier quoted context omitted.
Down Detector largely seems to track daily workplace usage patterns more than meaningful outages.
Which is great to detect common issues across many companies. For example, clicking on the cards shows that many of them are related to "network connection".
H&R Block's page there has this as the most recent comment, from an hour ago:
> My sister was able to have a bank pull money off her card yes her old card dunno which bank ill find out in bit
Two hours ago:
> I went to atm and thought I was crazy my pin wasnt working.
These reports are entirely useless.
Re: Slack is down
#463Re: Slack is down
#464> Customers may have trouble connecting or using Slack I can't stand how marketing speak pervades every sphere of the world. Their entire system is offline (inconvenient certainly, but it happens) and they can't bring themselves to say "Slack is down. We're working on it and will be back ASAP." or something similar. Instead we may have trouble.
The funniest part to me is that their status page still says "Uptime for the current quarter: 100%". These uptime messages are so BS. Heroku reports 6 9s of uptime for this month, even though their own status page shows multiple days with incidents >6 hours
Re: Slack is down
#465At GitLab our fallback from Slack is Zoom https://about.gitlab.com/handbook/communication/#emergency-c... I'm posting this because I found a lot of people don't know that Zoom includes a complete chat client that includes channels. And #HugOps to the engineers at Slack working on this. I appreciate that they posted a periodic update even when there was no news to report: "There are no changes to report as of yet. We'…
Re: Slack is down
#466* lots of users are coming back to work after the holidays today
* lots of users take the holidays off and fully disconnect
* significant new users added in 2020, with so many teams going remote
Sounds like a possible recipe for infra scaling issues and/or cascading failures to me
Re: Slack is down
#467Earlier quoted context omitted.
I'm quite surprised that you don't use Mattermost as a 'Slack fallback' at GitLab.
Indeed, if you champion FOSS, why would you recommend a proprietary piece of software as fallback?
That is one reason we did not go with Mattermost.
Re: Slack is down
#468> Customers may have trouble connecting or using Slack I can't stand how marketing speak pervades every sphere of the world. Their entire system is offline (inconvenient certainly, but it happens) and they can't bring themselves to say "Slack is down. We're working on it and will be back ASAP." or something similar. Instead we may have trouble.
Re: Slack is down
#469Earlier quoted context omitted.
It doesn't. The status page is currently showing information about the outage. And the 100% uptime number is probably still correct, since it's only been out for a couple of hours.
> And the 100% uptime number is probably still correct, since it's only been out for a couple of hours. It's listed as "Uptime for the current quarter"; if they mean that as "calendar quarter", i.e. since the start of the year, then we aren't even 100 hours into the quarter so we should be well below 100% by now.