Live data from Hacker News

Slack is down

status.slack.com

641–650 of 840 posts

Re: Slack is down

#641

I'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all…

RocketChat works pretty well for simple team comms. I have no idea if it can do XMPP and/or Matrix.

Re: Slack is down

#642

I'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all…

XMPP is supported by a large number of clients, but running a server and getting everyone on clients with comparable featuresets is a nightmare. It’s a cluster of disparate standards, and it’s overwhelming. I’m sure it’s doable if you have the time to invest, but it’s not straightforward if you’ve never done it before.

Matrix is pretty straightforward on the server side of things, but the client UX is invariably mediocre. Vector—the official client—exemplifies everything that is wrong with Electron apps. Slow, clunky, poor UI, poor platform integration. With the default home server, it can take seconds for a message to go through. At least it’s far more customizable than Slack; it has an option for everything, which, as a power user, I quite like.

I haven’t tried Mattermost, but it looks like some of the important features aren’t FOSS, at which point it’s just another Slack as far as I’m concerned. I’ll gladly pay for support, but for SSO? Meh, might as well stick with Slack; at least everyone and their dog knows how to use it. (This is, of course, an opinion that stems partially from ignorance; I haven’t actually tried Mattermost, and if I do, I might fall in love with it. But my time is limited, and I can only evaluate so many products in a day.)

Not that Slack is much better here: their threading system has so many UI/UX issues. Ever had a thread with hundreds of messages? For your own sanity, I hope you haven’t. Ever tried to send an image to a thread from iOS? It’s possible, but only by pasting the image into the text field; the normal attachment button isn’t available, and Share buttons in other apps can’t send to threads. And, of course, the recent uptime issues.

Re: Slack is down

#643
post #89

When I was at Uber, we noticed that most incidents are directly caused by human actions that modify the state of the system. Therefore, a large "backlog" of human actions that modify the system state have a much higher chance of causing an incident. My bet is that this incident is caused by a big release after a post-holiday "code freeze".

I would bet it's just the influx of traffic post holiday with systems that haven't been updated in so long maybe some annoying memory leaks have crept up and gone unnoticed or some other bad state that was exacerbated by return to work day for most NA folks. Code freezes were good at identifying bugs that only show up after long periods. Doubt anyone releasing big changes Monday morning.

[deleted]

Re: Slack is down

#644

Earlier quoted context omitted.

> And the 100% uptime number is probably still correct, since it's only been out for a couple of hours. It's listed as "Uptime for the current quarter"; if they mean that as "calendar quarter", i.e. since the start of the year, then we aren't even 100 hours into the quarter so we should be well below 100% by now.

You might be correct, but why would anyone care about quarter-to-date as opposed to a rolling quarter ending now? The latter would mean that an outage of X duration will always reduce this statistic by the same amount regardless of how close the nearest calendar quarter boundary is, which seems like a superior quality for such a statistic to have.

That would be a completely fair metric to publish, but it doesn't look like what Slack is publishing. Of course, it's possible that it is and it's just phrased somewhat poorly.

Re: Slack is down

#645

I'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all…

XMPP is supported by a large number of clients, but running a server and getting everyone on clients with comparable featuresets is a nightmare. It’s a cluster of disparate standards, and it’s overwhelming. I’m sure it’s doable if you have the time to invest, but it’s not straightforward if you’ve never done it before. Matrix is pretty straightforward on the server side of things, but the client UX is invariably medi…

Element (formerly Riot/Vector), has improved loads over the years, and the default matrix.org average send time is around 100ms these days rather than multiple seconds: https://matrix.org/blog/2020/11/03/how-we-fixed-synapses-sca... has details. I suspect you (and the parent) may be running off stale data.

That said, Element could certainly use less RAM, irrespective of Electron - and http://hydrogen.element.io is our project to experiment with minimum-footprint Matrix clients (it uses ~100x less RAM than Element).

Re: Slack is down

#646

I'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all…

IRC may be out these days, but at least deploying a small IRC server for the own team is really not that much effort anymore and doesn't incur that much ongoing maintenance work either.

Re: Slack is down

#647

While Slack is down, let's remind ourselves that it is not the end of the world. To their ops team, good luck in sorting out the root cause(s), to mitigating their re-occurrence, and to emerging the other side a stronger team. You've got this.

It is not end of the world if you are just using slack for intra team communications. However lot of the monitoring which alerts on slack and other automatic notifications are critical for many teams.

Critical systems dependent on another system are just as reliable as the third-party system; so this may be a good wake-up call for many.

Re: Slack is down

#648

I'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all…

I can only offer my own personal experience: Matrix has been working well for me for a couple years now. However, I probably have a more narrow use case than you're thinking of.

I run a small homeserver and use it to communicate with a group of about 20 friends. Most of them aren't "technical" people. We use it mostly for chatting and image/video sharing. We never use live calling (audio or video).

There have been a few bugs in the mobile apps, but for the most part, everything has been working fine.

The biggest issue is the UX. It's not as polished as the big players.

Re: Slack is down

#649

Earlier quoted context omitted.

XMPP is supported by a large number of clients, but running a server and getting everyone on clients with comparable featuresets is a nightmare. It’s a cluster of disparate standards, and it’s overwhelming. I’m sure it’s doable if you have the time to invest, but it’s not straightforward if you’ve never done it before. Matrix is pretty straightforward on the server side of things, but the client UX is invariably medi…

Element (formerly Riot/Vector), has improved loads over the years, and the default matrix.org average send time is around 100ms these days rather than multiple seconds: https://matrix.org/blog/2020/11/03/how-we-fixed-synapses-sca... has details. I suspect you (and the parent) may be running off stale data. That said, Element could certainly use less RAM, irrespective of Electron - and http://hydrogen.element.io is ou…

It has, and I’ve been using it since its early days. I still use it. It’s still terrible, just slightly less terrible. And, no, messages don’t consistently send in 100ms on the default home server; there are regularly disruptions that cause significant delays, sometimes as much as 10-20sec. That’s a big problem for a federated chat platform.

Edit 1: I want to love it; the design is everything I could ever hope for in a chat platform. I even tried to contribute to Vector, but it was such a mess that I eventually gave up.

Edit 2:

> That said, Element could certainly use less RAM, irrespective of Electron - and http://hydrogen.element.io is our project to experiment with minimum-footprint Matrix clients (it uses ~100x less RAM than Element).

I'm not sure why this is a priority. Techies complain about RAM usage a lot, but if we have to choose between performance+power and a small memory footprint, we're going to choose the former almost every time. Take Telegram, for example: they have a bunch of native clients that perform amazingly well, although they do gobble RAM. Most of my technical friends use it as their primary social platform. It's not without issues, but it's really hard to go from something like Telegram Desktop or the Swift-based macOS Telegram client to Vector. And those clients aren't made by large teams--most (all?) first-party Telegram clients are each maintained by a single developer, if I'm not mistaken.

Re: Slack is down

#650
post #606

Earlier quoted context omitted.

I'm actually more confused after reading that. I assumed that you meant that tested in production on purpose, but it sounds , at a skim, like they do non-prod testing environments - in fact, it looks like they've gone to having multiple beta environments of every service?

My understanding is that they have a "tenancy" variable in every service call which can take a different code path. They seem to only have one environment for everything and do tests/experiments at code level based on this variable.

Ah, thanks; that explains it nicely
Post reply on HN