Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A terrible, horrible, no-good, very bad day at Slack

slack.engineering

Post reply on HN
Print view View original

101–110 of 282 posts

Re: A terrible, horrible, no-good, very bad day at Slack

#101

Post by k__ »

xtracto wrote: post #99
k__ wrote: post #85

After I used Discord in different contexts for months now (and Slack for years), I can't understand why someone willingly chooses Slack. It's the Atlassian of chat tools. Horrible performance and bad usabillity.

Threaded comments. I tried using Discord with a group of 4 people for real job and we missed threaded comments. Also the ability to draw on screen while screen sharing. So simple yet so useful.

Interesting.

I personally didn't like threads and this is the first time someone mentions them favorably.

Re: A terrible, horrible, no-good, very bad day at Slack

#102

Post by fimbulvetr »

jrockway wrote: post #82
vecio wrote: post #5

What's the differences of using HAProxy or Envoy between using the cloud load balancers of AWS or Google Cloud?

I've found that the cloud load balancers lag behind the state of the art in features and that their assumptions and configurations can be pretty brittle. I haven't used Amazon's ALB, but with the legacy ELB, they can't speak ALPN. So that means, if you use their load balancer to terminate TLS, you can't use HTTP/2. Their automatic certificate renewal silently broke for us as well; whereas using cert-manager to renew…

I agree with you that we still have some ways to go with getting LB right, especially WRT to K8S. I think one of the problems is that it seems like every different app is a snowflake with different requirements, so all of these libraries try to be the jack of all trades, leaving the mastery to custom scripts (if it's even obtainable).

For instance: https://github.com/kubernetes-sigs/aws-alb-ingress-controlle...

Also, and you probably already know about this, but it's true that ingress won't create the record automatically for you - but external-dns ( https://github.com/kubernetes-sigs/external-dns ) will - with the correct annotations (pretty simple), external-dns will watch for changes to ingress and publish the dns records on R53 (and many other DNS providers) for you. It works really well for us, even when the subdomain is shared with other infrastructure not managed by itself.

Re: A terrible, horrible, no-good, very bad day at Slack

#103

Post by Jgrubb »

sudhirj wrote: post #31

This is one of the biggest arguments I see for serverless (AWS Lambda + DynamoDB) or at least managed PaaS systems (Google App Engine, Heroku with RDS or CloudSQL). These systems may seem to cost more for some workload curves (or might even be cheaper for your curve), but the difference is worth it because you're paying for specialized 24/7 dev-ops teams whose only job is to keep these systems running smoothly, and b…

I work for a PaaS.

- The ideal fit for any hosting PaaS is a company who has a large hosting and infra footprint but for whom the technology is _not_ the core competency of the business. Slack is very much better off running their own systems with their own people.

- As someone who deals with customers every day I can tell you that yes - we know our platform specifically and how the internet works generally better than almost 100% of our customers, but we do not know _your application_ at all.

- Many problems in The Cloud are the result of application developers not understanding that there are performance differences between localhost and The Cloud, specifically around IO, or that there are performance differences between the Cloud at 1x and The Cloud at 10x with everything. Systems can run smoothly and applications can still kill themselves because of the interaction between the two.

Re: A terrible, horrible, no-good, very bad day at Slack

#104

Post by anderspitman »

k__ wrote: post #101
xtracto wrote: post #99

Earlier quoted context omitted.

Threaded comments. I tried using Discord with a group of 4 people for real job and we missed threaded comments. Also the ability to draw on screen while screen sharing. So simple yet so useful.

Interesting. I personally didn't like threads and this is the first time someone mentions them favorably.

I missed them in discord just yesterday

Re: A terrible, horrible, no-good, very bad day at Slack

#107

Post by SamuelAdams »

kd5bjo wrote: post #2

The title is an allusion to a popular children’s book[1]. I’m assuming that an automated algorithm pulled the “very;” hopefully the mods will consider restoring it. [1] https://en.wikipedia.org/wiki/Alexander_and_the_Terrible,_Ho...

There's also a film based on the book.

https://en.wikipedia.org/wiki/Alexander_and_the_Terrible,_Ho...

Re: A terrible, horrible, no-good, very bad day at Slack

#108

Post by yjftsjthsd-h »

ryan_lane wrote: post #84
gchamonlive wrote: post #33

Earlier quoted context omitted.

You can't assume that, these costs, specifically the server less stack at AWS, scale up quite opaquely. You can put billing alerts in place, but once you tie your infrastructure to one specific serverless vendor, even if you identify a harmful cost scale, you can't easily mitigate it. I am not saying serverless is expensive, all I am advocating is extensive planning and preparing before adopting any particular server…

Lock-in is one of the worst reasons to avoid serverless. If you're really putting in the effort to avoid lock-in, you're wasting engineering time that could be better spent on your product. AWS (or GCP, or whatever cloud provider you choose) has a number of really great products that ultimately save you time and help you get to market faster and better. It doesn't matter that you're locked-in, with slightly higher th…

I'd disagree pretty strongly. You may evaluate the tradeoffs and decide to go all-in on a single cloud (and you should do that analysis!), but it's far from a given. You're not wasting engineering time if the costs get out of hand and bankrupt you. Managed services may accelerate your time to market, but they do so at the expense of lock-in and your bill.

Context: I work at a startup that benefits enormously by avoiding AWS/GCP (for most cases) and renting cheap dedicated servers. It is context-dependent; our exact business doesn't benefit much from managed services and really needs big servers.

Re: A terrible, horrible, no-good, very bad day at Slack

#109

Post by anderspitman »

Seems to me that most of the problems came from the sheer scale Slack operates at. A single instance of a self-hosted chat application wouldn't require any of the load balancing infrastructure. But SaaS is really convenient. I wonder if there's a market for "local cloud" companies that operate out of your city and offer hosting of popular open source projects. The complexity would be much lower, and hopefully the reliability higher. Plus you get the benefit of lower latency. 90% of the people in my team's slack channels live with 5 miles of each other.

Re: A terrible, horrible, no-good, very bad day at Slack

#110

Post by rpdillon »

k__ wrote: post #96
jrockway wrote: post #89

Earlier quoted context omitted.

As someone who uses Discord, I can tell you it goes down too. My favorite part is looking at their status page, seeing that API response time is exceedingly high, and getting no updates from the team about whether or not they're fixing it.

I'm not saying Discord is perfect or "always up", it's just that their client UI is better structured and more responsive in most cases. I am on multiple OSS Discord servers with thousands of users, and it works just fine most of the time. I am on multiple Slack servers with just 10-20 users and it is unbearably slow.

I admined a multi-hundred user Slack org and now am part of a multi-thousand user slack org and I've never experienced (or heard about) the slowness that you're describing. Do you have a feel for whether it was client-side or server-side?
Post reply on HN