Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Slack is down

status.slack.com

Post reply on HN
Print view View original

401–410 of 840 posts

Re: Slack is down

#401

Post by lsaferite »

blntechie wrote: post #332

I have always had this fantasy thinking of what happens when outages of one of these major service never come back online i.e. in this outage Slack loses info of all the accounts, users, messages etc. How would people react? What would engineers do to recover? I always found that idea fascinating. Imagine Google saying tomorrow that they lost all the accounts and emails. What kind of impact the world will have?

This should actually be part of your Disaster Recovery plan. You should have at least some plan for the loss of all of your service providers. Even if that plan is to sit in the corner and cry (j/k).

Re: Slack is down

#402

Post by yjftsjthsd-h »

aeyes wrote: post #229
exhaze wrote: post #175

Earlier quoted context omitted.

> run it in a beta environment Every Uber/ex-Uber engineer is nervously chuckling at this comment right now

For those that don't know what this comment is about: https://eng.uber.com/multitenancy-microservice-architecture/

I'm actually more confused after reading that. I assumed that you meant that tested in production on purpose, but it sounds, at a skim, like they do non-prod testing environments - in fact, it looks like they've gone to having multiple beta environments of every service?

Re: Slack is down

#403

Post by 013a »

gruez wrote: post #354
jjice wrote: post #255

Earlier quoted context omitted.

I didn't realize that Discord has way more active users than Slack. I'm glad, Discord is a fantastic service in my experience. It's a shame they got shoe horned into a mostly gaming oriented service. I've never had a class or worked somewhere where Discord was a considered solution instead of Slack, but I can't think of anything that Slack does better (in my experience). In general, I think Discord has the best audio…

>I didn't realize that Discord has way more active users than Slack Keep in mind you're comparing daily active users vs monthly active users. I'd guess most slack users are online weekday for pretty much the entire day (because it's for work and your boss expects you to be online), whereas a good chunk of discord users are only logging in a few hours a week when they're gaming.

At 12:00pm EDT on a workday:

Minecraft official server: 190k online users. | Fortnite official server: 180k online users. | Valorant official server: 170k online users. | Jet's Dream World (community): 130k online users. | CallMeCarson server (YouTuber): 100k online users. | Call of Duty official server: 90k online users. | Rust (the game) official discord: 80k online users. | League of Legends official server: 60k online users. | Among Us official server: 50k online users.

Their scale is insane. Even with their usage spiking during after-hours gaming in major countries, their baseline usage at every hour of the day, globally, makes it one of the most used web services ever created.

Slack's DAU and MAU numbers are probably pretty close to one-another. Discord's MAU/DAU ratio is probably bigger than Slack's. That just means that Discord is, again, solving a harder problem; they have much bigger (and more unpredictable) spikes in usage than Slack. Yet, its a far more stable and pleasant product.

Re: Slack is down

#404

Post by blntechie »

codingdave wrote: post #384
blntechie wrote: post #332

I have always had this fantasy thinking of what happens when outages of one of these major service never come back online i.e. in this outage Slack loses info of all the accounts, users, messages etc. How would people react? What would engineers do to recover? I always found that idea fascinating. Imagine Google saying tomorrow that they lost all the accounts and emails. What kind of impact the world will have?

That scenario is what Disaster Recovery plans are for. Every large company I've worked for has had recovery plans in place, including scenarios as disturbing as "All data centers and offices explode simultaneously, and all staff who know how it all works are killed in the blasts." You not only have backups in place, you have documentation in place, including a back-up vendor who has copies of the documentation and ca…

That’s some really good thoughts on DR planning. I have never thought DR to be to such an extent.

How many companies really plan for an event where their entire infrastructure goes offline and their entire team gets killed? Does even companies like Google plan for this kind of event?

Re: Slack is down

#407

Post by Scandiravian »

vlod wrote: post #365
Scandiravian wrote: post #244

Earlier quoted context omitted.

I just finished an 5 hour debugging session on what turned out to be several cascading bugs in one of the older systems at my company Can't deploy the fix, because - developers trigger deployments through slack and I don't have access to the underlying deployment system - infrastructure guys who have access aren't responding to my emails

Sounds like a great opportunity to work on not having Slack as the only trigger mechanism. :) Or a least document how to call the slack bot manually. (assuming it's just a http endpoint)

I agree 100%! Though I think it might be dangerous to prepare for the "last disaster". It'll be some other system breaking next time, so I think we instead should identify what systems that do not have some kind of redundancy and determine blast radius of those crashing

I'm good at not panicking about things I can't change, but I worry about some of my colleagues who find it difficult to not have control in these situations

I can't do anything to help them at the moment, so for now I'm heading to my couch with my analogue book :)

Re: Slack is down

#408

Post by marricks »

Does Google use slack? Wanted to start my year with some extra strength tinfoil and it'd just be great if the day a unionizing initiative started the major way workers could talk about said unionizing initiative went down.

EDIT: according to a random quora post they do, so keep the tinfoil out!

Re: Slack is down

#409

Post by danepowell »

tshaddox wrote: post #380
ARandomerDude wrote: post #281

> Customers may have trouble connecting or using Slack I can't stand how marketing speak pervades every sphere of the world. Their entire system is offline (inconvenient certainly, but it happens) and they can't bring themselves to say "Slack is down. We're working on it and will be back ASAP." or something similar. Instead we may have trouble.

Why do you consider that to be "marketing speak?" It appears to be concise, direct, and accurate. The phrase "Slack is down," even if true by some interpretations (it hasn't been "completely down" from what I have seen), is imprecise and informal.

There's a wide gulf between "some customers may have trouble using Slack" and "most/all customers are completely unable to use Slack". Putting aside formality, I'd say "Slack is down" is in fact more accurate here (assuming that it is true that most users can't use it, which is true for our company at least).

Re: Slack is down

#410

Post by MattGaiser »

criley2 wrote: post #390
_6fmb wrote: post #343

Earlier quoted context omitted.

What an overwrought headline, the employee in question has already been fired.

It's weird that you describe the headline as "overwrought" and call the person an "employee" when the headline is more accurate than you. This was an executive, not just an employee. That's a huge distinction and I can't help but think you intentionally downgraded his position to cover-up his behavior. "Just an employee" "Not a big deal" But when you read the allegations, they seem like a very big deal that an execut…

Not only that, but this line stuck out to me.

> and other employees have been placed on administrative leave until the investigation is complete.

Zoom at least suspects he did not act alone.

Post reply on HN