Continuous Deployment at Instagram
engineering.instagram.com
Continuous Deployment at Instagram
1–10 of 94 posts
Re: Continuous Deployment at Instagram
#2What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
Re: Continuous Deployment at Instagram
#3What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
A lot of it depends on the particulars of your database system. But there are certainly tools/solutions that exist, and essentially they all boil down to the same pattern: migration scripts should be executed exactly once in order. How this is done varies.
I do a lot in the .NET/SQL Server world, and my tool of choice is one that I wrote: http://josephdaigle.me/2016/04/03/introducing-horton.html. Conceptually, what this tool does could work for any RDBMS.
Re: Continuous Deployment at Instagram
#4What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
I work in a company that does several deployments a day, and has a giant database. The short answer is you design around it. If you can do something without majorly changing your data you do it. Another common thing is to deploy the code, but then to add a "feature toggle" so you just turn the code on when the data is ready. Basically, figure out how you can refuel while in the air.
Re: Continuous Deployment at Instagram
#5What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
I've used Flyway for db migrations and it works well.
Re: Continuous Deployment at Instagram
#6Is their test suite just unit tests, or does it include user-acceptance testing?
Re: Continuous Deployment at Instagram
#7We do something very similar at JotForm. The hardest part has always been the"keeping it fast" part. The deployment should take less than minute otherwise developers spend too much time waiting to see if the commit went live without problems.
The solution is usually to run things in parallel. Examples: Run tests in parallel. Keep things in separate repos and push them into separate folders in the app servers.
Re: Continuous Deployment at Instagram
#8What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
Beyond a certain size (basically, once the time the migration will take because of the size of the data it applies to is too large), migrations are a heavy investment - in elapsed time, I/O, and so forth. As such, they are planned to a degree that CD probably isn't the solution for it (for example, you probably can only have one migration in flight at a time).
They aren't done live as a single big process that have the potential to lock all queries/updates over their execution, but rather as a set of smaller steps that don't lock.
Facebook has spoken about its online schema change process before - https://www.facebook.com/notes/mysql-at-facebook/online-sche... and its follow-up at https://www.facebook.com/notes/mysql-at-facebook/online-sche... for example, and I'm sure elsewhere.
Most people using MySQL would potentially first use something like https://www.percona.com/doc/percona-toolkit/2.1/pt-online-sc... instead of trying to create their own.
The same principles apply to other data stores that have more rigid schemas.
Re: Continuous Deployment at Instagram
#9What are the best practices for database migrations when trying to setup continuous deployment? Are there any existing tools/solutions that solve/simplify the problem? This is the issue that is almost always missing in articles/tutorial about CD
I work in a company that does several deployments a day, and has a giant database. The short answer is you design around it. If you can do something without majorly changing your data you do it. Another common thing is to deploy the code, but then to add a "feature toggle" so you just turn the code on when the data is ready. Basically, figure out how you can refuel while in the air.
> The short answer is you design around it.
Sorry but that seems like a pretty crap answer.
The answer to "what tools allow you to manage database migrations with CD" should not be "don't do database migrations with CD" or "roll your own toggling features."
Re: Continuous Deployment at Instagram
#10Is their test suite just unit tests, or does it include user-acceptance testing?
[deleted]