Slack – Degraded service affecting multiple features • Hacker News Board

Re: Slack – Degraded service affecting multiple features

#171

Post by orwin » Fri, Jun 28, 2019, 2:49 PM UTC

untog wrote: post #142

alt_f4 wrote: post #130

Earlier quoted context omitted.

Come on, in the 90s and early 00s, volunteers ran and managed IRC networks scaling up to hundreds of thousands of users. This was back when hardware and bandwidth were much weaker and 100x more expensive. It's really not that hard.

Yes, volunteers ran these things. How many? Because you're going to need to have paid employees rather than volunteers running your servers. Is it going to be more reliable and cost effective than just paying a third party to run a service for you?

If you're a big enough company (more than 60 people if computer are your means of production, around a thousand if not), you should have your own server and a fallback on a CSP. Just pay some guy in India/South Africa to manage all your internal services and architecture. If they are sensible and you need to trust the guy, just pay him well and hire locally.

I can set you up a resillient architecture and 3 services (communication, documentation, filesharing) in less than two weeks (let's say two weeks and a half because security concerns/limitations and all that stuff). Add one week and you will have mail server, VOIP communication, log conservation and monitoring, backup images on CEPH, vnc service for "easy" maintainance and a small documentation explaining the few action needed to maintain the monster for two years. Initial cost for a company: 15 to 25k$, + 1 hour and ~100$ a month.

Re: Slack – Degraded service affecting multiple features

#172

Post by beatgammit » Fri, Jun 28, 2019, 2:50 PM UTC

Medicalidiot wrote: post #139

valerij wrote: post #135

Earlier quoted context omitted.

i stand by my opinion that "lets create one chat room and invite hundreds of thousands of people into it" is a bad idea regardless of what technology implements the chat room

I respect your opinion, because even if an application could handle 200k users at once was able to be run on a MBP without serious issue, it's still gauche to forcibly invite 200k people in your company to a chat.

Slack is designed for business chat, and many businesses have thousands or more employees. Slack should be able to handle any features it provides, and @everyone is a feature. I don't know how they've implemented things or what the specific issue is, but it should at least have blocks in place that prevent stupid things. Maybe channel sizes should be limited to 1000 people or whatever their testing shows it can handle. Idk.

Software designed for large numbers of people shouldn't go down because one of those people does something stupid.

Re: Slack – Degraded service affecting multiple features

#173

Post by manquer » Fri, Jun 28, 2019, 2:50 PM UTC

pmlnr wrote: post #124

untog wrote: post #99

Earlier quoted context omitted.

"just running an IRC server" is... a lot. Or can be. Who is running it? Does it scale well? How is login synced with corporate logins? What about mobile devices? It's not "better" than email, it's different. Real time. It has a million and one service integrations, which IRC doesn't. Point is, it's really not all that crazy to just pay a third party to manage all this stuff for you. You might imagine the alternative…

> Does it scale well? It's IRC. On modern hardware. It's practically infinite. > it has a million and one service integrations, which IRC doesn't. There and were are things way before Slack, eg https://loqi.me/

Features is not the only thing, does it have well funded sales team and strong corporate support ? Slack itself did support IRC till recently, the product is not just a sum of its features

Re: Slack – Degraded service affecting multiple features

#174

Post by jordanbeiber » Fri, Jun 28, 2019, 2:50 PM UTC

iamnotacrook wrote: post #161

valerij wrote: post #135

Earlier quoted context omitted.

i stand by my opinion that "lets create one chat room and invite hundreds of thousands of people into it" is a bad idea regardless of what technology implements the chat room

If you want to tell everyone in your company something, you could email them I guess. Is it the fact that it's a "chat room" which means you expect it not to work? What's best, here? SMS? Email? Putting something on the website and making everyone check the website once a day at noon? Tell one in ten people and get them to "pass it on"?

Most companies today have a decent intranet site for these things.

Re: Slack – Degraded service affecting multiple features

#175

Post by Wowfunhappy » Fri, Jun 28, 2019, 2:51 PM UTC

pmlnr wrote: post #165

untog wrote: post #148

Earlier quoted context omitted.

Obviously you can create IRC integrations for anything. But how many are installable with one click, like Slack integrations are? Having to write custom implementations is yet another extra task and point of failure compared to just paying someone to manage it all for you.

another "but". First "but it doesn't scale". Now "but integrations". There will always be another "but", and there will always be a link on the internet showing that it's been done already. "but shiny web gui" - https://thelounge.chat/ "but mobile app" - https://github.com/MCMrARM/revolution-irc "but it's complicated" - https://www.mirc.com/ since 1995 IRC does have limits. Some. One is that to make it "modern" it ne…

How do you make it possible for people to view the IRC channel history when they were disconnected? I know you can have a bot that saves a log, but can it be made seamless?

I think it's really sad how the world has migrated away from an open protocol to a closed one, but I remember IRC having some very real annoyances, well beyond stuff like integrations.

Re: Slack – Degraded service affecting multiple features

#176

Post by pmlnr » Fri, Jun 28, 2019, 2:51 PM UTC

manquer wrote: post #173

pmlnr wrote: post #124

Earlier quoted context omitted.

> Does it scale well? It's IRC. On modern hardware. It's practically infinite. > it has a million and one service integrations, which IRC doesn't. There and were are things way before Slack, eg https://loqi.me/

Features is not the only thing, does it have well funded sales team and strong corporate support ? Slack itself did support IRC till recently, the product is not just a sum of its features

> does it have well funded sales team and strong corporate support

Get 2 sysadmins. 2 because of redundancy. Pay them to maintain your service.

"well funded sales team and corporate support". F that. Trust your employees.

Re: Slack – Degraded service affecting multiple features

#177

Post by phtrivier » Fri, Jun 28, 2019, 2:51 PM UTC

lqet wrote: post #74

Can someone explain to me why any company should outsource something as critical as internal communication to a company that is 5 years old? The video messaging left aside, is it really that better than time-tested solutions like email or just running an IRC server?

* search engine that just work

* push-notifications that just work

* client available for enough people

* copy / pasting images that just work (the'll never admit it, but animated gif is their killer feature)

* stable-enough infrastructure for something that costs you 15€/month/user not taken from your "salary" budget (often a different and substantially less taxed part)

* and their outages are funny (receiving messages twice is arguably better than not receiving message)

Yes, it's because it "just kinda works for what it costs, and you can forget about it most of the time and do your job."

This seems to be characterized as "laziness" by some people, a point I honestly don't understand (I though lazyness was a virtue in our circles.)

That being said, there are legitimate reasons to not use it:

* software licence

* data property

* technical implementation of the native clients

* general distrust of centralized solutions

* cost of having IM at all (see Newport's whole publishing career)

At least, it means there's an opportunity to create a company that handles decentralized user-friendly open-source standard-based data-respectful attention-preserving uber-reliable chat system. HN users would be a market.

Re: Slack – Degraded service affecting multiple features

#178

Post by derefr » Fri, Jun 28, 2019, 2:51 PM UTC

Karunamon wrote: post #164

untog wrote: post #142

Earlier quoted context omitted.

Yes, volunteers ran these things. How many? Because you're going to need to have paid employees rather than volunteers running your servers. Is it going to be more reliable and cost effective than just paying a third party to run a service for you?

For something like IRC? Yes. The service is dead simple and one server can handle thousands of users with modest hardware and without breaking a sweat. A completely unfamiliar sysadmin could set it up in an hour or less. Since it's probably internal, there's little need for public services (nick/channel registration, etc).

Most of how I use Slack, and how everyone I know uses Slack, involves 1. sending messages to people who are “away” to get their attention/loop them into a conversation; and 2. people reading back up in history for context when they come online.

Now, modern IRCds might have features I’ve never experienced on raw IRC... but if they don’t, I don’t suppose you’re suggesting that this server also run thousands of bouncer bots in order to provide this contextual history feature?

(Also, whence push notifications?)

Re: Slack – Degraded service affecting multiple features

#179

Post by bdonlan » Fri, Jun 28, 2019, 2:52 PM UTC

cabaalis wrote: post #160

nielsole wrote: post #5

Best thing is that some messages go through despite returning 500s. This leads to our integrations spamming our companies channels with redundant messages while retrying. :/

Ouch, that one hurts. I don't know slack API but the contract expectation in the call should absolutely be "500 means the message failed to send" Tells me there's some subsystem throwing the error, not the actual messaging (or, if intermittent, something wrong with the distribution.) What do you want to bet it's some kind of new analytics?

It's not possible to always ensure that you report whether an operation succeeded. Consider a timeout - you waited 5 seconds for the backend to respond and heard nothing - was it, or will it be successful? You don't know.

Typically a 500 should be considered as a "may or may not have succeeded", and steps taken to deduplicate operations if necessary.

Re: Slack – Degraded service affecting multiple features

#180

Post by pmlnr » Fri, Jun 28, 2019, 2:52 PM UTC

Wowfunhappy wrote: post #175

pmlnr wrote: post #165

Earlier quoted context omitted.

another "but". First "but it doesn't scale". Now "but integrations". There will always be another "but", and there will always be a link on the internet showing that it's been done already. "but shiny web gui" - https://thelounge.chat/ "but mobile app" - https://github.com/MCMrARM/revolution-irc "but it's complicated" - https://www.mirc.com/ since 1995 IRC does have limits. Some. One is that to make it "modern" it ne…

How do you make it possible for people to view the IRC channel history when they were disconnected? I know you can have a bot that saves a log, but can it be made seamless? I think it's really sad how the world has migrated away from an open protocol to a closed one, but I remember IRC having some very real annoyances, well beyond stuff like integrations.

The answer is in at the bottom of my previous entry. It's called an IRC bouncer.