Live data from Hacker News

Slack’s migration to a cellular architecture

slack.engineering

1–10 of 265 posts

Re: Slack’s migration to a cellular architecture

#4
post #2

So they used a feature built into a load balancer to gracefully drain traffic from specific availability zones? Odd that a feature found in load balancers from the last 25 years is a blog post worthy thing.

The other bit is separating the service into isolated cells so issues in one don't affect dependent services everywhere like they had experienced before.

But yeah any good SRE could point this out years ago.

Re: Slack’s migration to a cellular architecture

#5
post #2

So they used a feature built into a load balancer to gracefully drain traffic from specific availability zones? Odd that a feature found in load balancers from the last 25 years is a blog post worthy thing.

That seems like a shallow dismissal. In a distributed system, making sure that sub requests are handled across distributed nodes within the local AZ, and correctly draining traffic from AZs with partial component service outages, is not as trivial as 'using a feature built in to a load balancer'.

Re: Slack’s migration to a cellular architecture

#7
post #4
post #2

So they used a feature built into a load balancer to gracefully drain traffic from specific availability zones? Odd that a feature found in load balancers from the last 25 years is a blog post worthy thing.

The other bit is separating the service into isolated cells so issues in one don't affect dependent services everywhere like they had experienced before. But yeah any good SRE could point this out years ago.

Just odd a company worth billions and billions of dollars is just now discovering HA models standard since the 90s. Can expand the Clos network architecture to these distributed service applications too. But judging by Slack's client quality, mature concepts such as those must be new to them.

Re: Slack’s migration to a cellular architecture

#8
post #2

So they used a feature built into a load balancer to gracefully drain traffic from specific availability zones? Odd that a feature found in load balancers from the last 25 years is a blog post worthy thing.

Close but I don't think it's quite 25 years! I added graceful draining to Apache httpd's mod_proxy and mod_proxy_balancer either in 2003 or 2004, and at the time I'm nearly certain it was the first software load balancer to have the feature, and it wasn't available on the hardware load balancers of the time that I had access to ... though I later learned that at least BigIP load balancers had the feature.

At the time, we had healthy debates about whether the feature was useful enough to justify additional complexity, and whether there could be cases where it would backfire. To this day, it's an underused feature. I still regularly run into customers and configurations that cause unnecessary blips to their end-users, so it's nice to see when people dig in and make sure that the next level of networking is working as well as it can.

Post reply on HN