Great article. I had never heard of dh-virtualenv but will be looking into it.
How has your experience with Ansible been so far? I have dabbled with it but haven't taken the plunge yet. Curious how it has been working out for you all.
How We Deploy Python Code
71–80 of 129 posts
Re: How We Deploy Python Code
#72Back when I was doing Python deployments (~2009-2013) I was:
* Downloading any new dependencies to a cached folder on the server (this was before wheels had really taken off) * Running pip install -r requirements.txt from that cached folder into a new virtual environment for that deployment (`/opt/company/app-name/YYYY-MM-DD-HH-MM-SS`) * Switching a symlink (`/some/path/app-name`) to point at the latest virtual env. * Running a graceful restart of Apache.
Fast, zero downtime deployments, multiple times a day, and if anything failed, the build simply didn't go out and I'd try again after fixing the issue. Rollbacks were also very easy (just switch the symlink back and restart Apache again).
These days the things I'd definitely change would be:
* Use a local PyPi rather than a per-server cache * Use wheels wherever possible to avoid re-compilation on the servers.
Things I would consider:
* Packaging (deb / fat-package / docker) to avoid having any extra work done over per-machine + easy promotions from one environment to the next.
Re: How We Deploy Python Code
#73For installing using .deb files, how are db migrations handled. Our deployment system handles running django migrations by deploying to a new folder/virtualenv, running the migrations, then switching over symlinks.
I vaguely remember .deb files having install scripts, is that what one would use?
Re: How We Deploy Python Code
#74http://pythonwheels.com/ solves the problem of building c extensions on installation.
Pair this with virtualenvs in separate directories (so that "rollback" is just a ssh mv and a reload for whatever supervisor process) and you get to skip the mess of building system packages. Also, are there seriously places that don't run their own PyPI mirrors? Places that have people who understand how to integrate platform-specific packages but can't be bothered to deploy one of the several PyPI-in-a-box systems…
> you get to skip the mess of building system packages.
Only in cases where you don't have wheels depending on external libraries. If you do, you should still package with the right dependency constraints. Otherwise you can install a wheel which does not work (because of missing .so)
Re: How We Deploy Python Code
#75Their reason for dismissing Docker are rather shallow, considering that it's pretty much the perfect solution to this problem.
Their first reason (not wanting to upgrade a kernel) is terrible considering that they'll eventually be upgrading it anyways.
Their second is slightly better, but it's really not that hard. There are plenty of hosted services for storing Docker images, not to mention that "there's a Dockerfile for that."
Their final reason (not wanting to learn and convert to a new infrastructure paradigm) is the most legitimate, but ultimately misguided. Moving to Docker doesn't have to be an all-or-nothing affair. You don't have to do random shuffling of containers and automated shipping of new images—there are certainly benefits of going wholesale Docker, but it's by no means required. At the simplest level, you can just treat the Docker contain as an app and run it as you normally would, with all your normal systems. (ie. replace "python example.py" with "docker run example")
Re: How We Deploy Python Code
#76Great article. I had never heard of dh-virtualenv but will be looking into it. How has your experience with Ansible been so far? I have dabbled with it but haven't taken the plunge yet. Curious how it has been working out for you all.
Ansible works well for us, although we use it in a somewhat different way than most folks. We previously wrote about our approach here, if you're curious: https://nylas.com/blog/graduating-past-playbooks
Re: How We Deploy Python Code
#77Earlier quoted context omitted.
You need those things up to build the package not to install it
Ah I misunderstood the article. I just package my application source in a tar during deployment. I thought that's what most people do.
Using a native package gives so much more power - you define that the package relies on python, and maybe a mysql or postgres client, redis, whatever it needs, and then just install the one package, and let apt/dpkg handle the dependencies.
I'm a big fan of using the config-package-dev package from DebAthena to build config packages, which allow for about 99.9% of Debian server setup to be defined in Debian packages.
Re: How We Deploy Python Code
#78Seems this method wouldn't work as well if you have external clients you deploy for. I'd use Docker instead of doing this, just to be in a better position for an internal or external client deployment.
If you took this a step further and set up a debian repo, then you could have your clients use that debian repo. I'm looking to do something pretty similiar, but RPMs. I found rpmvenv that seems to work in the same fashion. https://pypi.python.org/pypi/rpmvenv/0.3.1
Exactly this.
If a company wants to use Docker that's their choice, but I don't think its at all reasonable to insist on or only support that environment as a software vendor. If it works on Debian, give me a .deb or even better an Apt Repo to use.
Re: How We Deploy Python Code
#79Their reason for dismissing Docker are rather shallow, considering that it's pretty much the perfect solution to this problem. Their first reason (not wanting to upgrade a kernel) is terrible considering that they'll eventually be upgrading it anyways. Their second is slightly better, but it's really not that hard. There are plenty of hosted services for storing Docker images, not to mention that "there's a Dockerfil…
> (not wanting to upgrade a kernel) is terrible considering that they'll eventually be upgrading it anyways.
If they're running ubuntu 12.04 LTS they can keep the 3.2 kernel until late 2017. That's 2 more years. And they wrote "did not", so it was likely the situation months ago, not yesterday.
> (not wanting to learn and convert to a new infrastructure paradigm) is the most legitimate, but ultimately misguided
It depends on the amount of stuff they deploy. If they handle everything using Ansible (and from the list it looks like they do), then it's months of work to migrate to something else. They may need the right users / logging / secret management in the app itself, not outside of it.
Re: How We Deploy Python Code
#80For installing using .deb files, how are db migrations handled. Our deployment system handles running django migrations by deploying to a new folder/virtualenv, running the migrations, then switching over symlinks. I vaguely remember .deb files having install scripts, is that what one would use?
Depends on your environment, number of hosts, etc. really. You probably don't want to stick it into the same install script because:
- your app user doesn't need rights to modify the schema
- you need to handle concurrency of schema upgrades (what if two hosts upgrade at the same time?)
- if your migration fails it may leave you in the weird installation state and not restart the service
Ideal solution: deploy code which can cope with both pre-migration and post-migration schema -> upgrade schema -> deploy code with new features.