Live data from Hacker News

Deploys at Slack

slack.engineering

51–60 of 139 posts

Re: Deploys at Slack

#51

A few questions I have left unanswered: - does the deploy commander create the hotfixes or the engineers who authored the commits? - it seems that the deployment is fully automated, but engineers still have to be available in case of problems, does that impact productivity? - "Once we are confident that core functionality is unchanged", is there a particular metric to assert that? - how long does deployment take curr…

With PHP (What slack was using at one point for some of the services. I think everything uses Hack now which may still maintain a similar model). Switching directories can be mostly atomic.

PHP-FPM with opcaching doesn't need to access files once all the opcodes are cached (turn off file modification checks in production). When you move the directory, you will restart the service.

Unless a request hits a file that is rarely used and not cached, you should be not receive any errors moving the directories.

Re: Deploys at Slack

#52
It's interesting that atomic deploys weren't in from the start. That was one of the few deployment practices we really insisted on from day one at my own businesses, if only because the uncertainty you get from trying to trace problems where your system isn't in any known state makes it all but impossible to work systematically.

A related challenge where we've never really found a good solution is how to handle deploying updates atomically when both code and data model are changing. That is, we need to migrate both our application software and our database schema in some co-ordinated way.

In practice, this usually ends up being done in multiple stages, where during some intermediate part of the process we are actively maintaining both the old and new database structure and running both versions of relevant code, at some point in the process there will be a bulk conversion of existing DB data that was only in the old format to the new one, and then hopefully at the end we switch to reading only the new version, retire the old code, and if necessary remove the old DB contents that are no longer in use. Even then we probably still want to keep an implementation of our previous data API available that is reverse engineering data from the new format, just in case we have to wind back the application code due to some other problem.

I got tired just writing that, and it feels similarly dirty actually deploying it. How is everyone else handling this? Has anyone found a satisfactory way to migrate code and data forwards, and if necessary backwards, without timing or data loss issues? Controlled deployments of application code seem to be largely a solved problem with modern tools and a bit of common sense, but the database side of things doesn't seem to be nearly as clean, at least not with any of the strategies I've encountered so far.

[Edit: I see that while I was writing this, someone else has already raised a similar point elsewhere in the discussion and a few people have replied, but unfortunately only along the lines I mentioned here as well. This does not make me optimistic about finding a cleaner strategy, but further comments are still welcome.]

Re: Deploys at Slack

#53

A few questions I have left unanswered: - does the deploy commander create the hotfixes or the engineers who authored the commits? - it seems that the deployment is fully automated, but engineers still have to be available in case of problems, does that impact productivity? - "Once we are confident that core functionality is unchanged", is there a particular metric to assert that? - how long does deployment take curr…

With PHP (What slack was using at one point for some of the services. I think everything uses Hack now which may still maintain a similar model). Switching directories can be mostly atomic. PHP-FPM with opcaching doesn't need to access files once all the opcodes are cached (turn off file modification checks in production). When you move the directory, you will restart the service. Unless a request hits a file that is…

My point is that if there is any downtime for the switch, for example restarting a service, it's not atomic. A small percentage of failed requests can still be high in absolute terms for a company like Slack, so why not using a paradigm [1] where you have atomic switch? And also instant rollback.

[1] https://www.martinfowler.com/bliki/BlueGreenDeployment.html

Re: Deploys at Slack

#54
post #43

Fun to read, but there's a lack of detail here that I'd like to see. For example, this talks purely about code changes. However times a code change requires a database schema change (as mentioned above), different API's to be used, etc. In the percentage based rollout where multiple versions are in use at once, how are these differences handled?

For database schema changes, here is the standard practice: - You have version 1 of the software, supporting schema A. - You deploy a version 2 supporting both schema A and new schema B. Both versions coexist until the deployment iis complete and all version 1 instances are stopped. During all this time the database is still on schema A, this is fine because your instances, both version 1 and 2, support schema A. - N…

My company uses HBase currently for things on premise and we're moving to a mix of psql and BigTable in GCP. This is how we do things except all of our "schemas" are defined by the client so we just have to make sure that serialization/deserialization works correctly. With psql we might have to figure out a migration strategy, but for now we'll just be using it to store raw bytes.

Re: Deploys at Slack

#55
post #46

Earlier quoted context omitted.

Sleuth is a SaaS deployment tracker that pulls deployments from source repositories, feature flags, and other sources, in addition to pushes via curl. You can see Sleuth used to, well, track Sleuth at https://app.sleuth.io/sleuth [Disclaimer: am a Sleuth co-founder]

Is it possible to view the page you linked without creating an account? It redirects me to your landing page.

Sorry about that the live demo is at https://app.sleuth.io/sleuth/sleuth

Re: Deploys at Slack

#57

It's interesting that atomic deploys weren't in from the start. That was one of the few deployment practices we really insisted on from day one at my own businesses, if only because the uncertainty you get from trying to trace problems where your system isn't in any known state makes it all but impossible to work systematically. A related challenge where we've never really found a good solution is how to handle deplo…

I've never seen it solved. You either write and test migration scripts to roll it back or you restore from a backup. Idk what you do if you add a new column that's populated in the new version and you rollback. I guess this would be a good place to roll out as small of piece as you can and hope you don't find out it's busted a week later.

Re: Deploys at Slack

#58
> Even strategies like parallel rsyncs had their limits.

They don't really go into detail as to what limitations they hit by pushing code to servers instead of pulling. Does anyone have any ideas as to what those might be? I can't think of any bottlenecks that wouldn't apply in both directions, and pushing is much simpler in my experience, but I've also never been involved with deployments at this scale.

Re: Deploys at Slack

#59
post #45
post #23

This link has now been reposted 6 times in the past two weeks: https://news.ycombinator.com/item?id=22816645 https://news.ycombinator.com/item?id=22729766 https://news.ycombinator.com/item?id=22801191 https://news.ycombinator.com/item?id=22784712 https://news.ycombinator.com/item?id=22720028 https://news.ycombinator.com/item?id=22806810

That's an indicator of interest. I actually emailed one of the submitters to repost the article for that reason. (Yes, we're thinking about software to detect cases like this.) On HN, a submission doesn't count as a dupe unless it has had significant attention. This is in the FAQ: https://news.ycombinator.com/newsfaq.html .

Fair enough, only reason I noticed is because I was actually going to post this link yesterday but did a search to make sure sure I wasn't reposting.

Plus they didn't get much traction anyway, so I wrongly assumed there wasn't interest.

Know for the future now!

Re: Deploys at Slack

#60
I'm kind of surprised they don't have a branch-based staging. Every place I've worked at has evolved in the direction of needing the ability to spin up an isolated staging environment that was based on specific tags or branches.
Post reply on HN