Python Deployment Anti-Patterns
hynek.me
Python Deployment Anti-Patterns
1–10 of 125 posts
Re: Python Deployment Anti-Patterns
#2Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist.
Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment process. Trying to shoe-horn the finer deployment steps (git checkout, tarballing, symlinks, building the virtualenv, etc) into Chef was a nightmare every time I tried. Those tasks are better suited for Fabric's imperative design. Plus you can just run any Chef commands from Fabric itself, or use something like pychef for finer grained control. It's a win/win.
Re: Python Deployment Anti-Patterns
#3Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist. Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment proces…
> Those tasks are better suited for Fabric's imperative design.
Yes they are, but IMHO not on the target servers.
I use Fabric to build DEBs that get deployed by Puppet. I prefer to have no build tools on target servers, YMMV.
Re: Python Deployment Anti-Patterns
#4As a python dev who deploys a lot of software, I found this article to be wonderfully helpful and informative, and a good reflection of current best practices.
Summary of the deployment tools mentioned:
- Manage remote daemons with supervisord
http://supervisord.org/
- Manage python packages with pip (and use `pip freeze`)
http://pypi.python.org/pypi/pip
http://www.pip-installer.org/en/latest/requirements.html
- Manage production environments with virtualenv
http://www.virtualenv.org/
- Manage Configuration with puppet and/or chef
http://puppetlabs.com/
http://www.opscode.com/chef/
- Automate local and remote sys admin tasks with Fabric
http://fabfile.org
- Don't restrict yourself to old Python versions to appease your tools / libs.
- Strongly consider rolling your own DEB/RPMs for your Python application.
- Celery for task management
http://celeryproject.org/
- Twisted for event-based python.
http://twistedmatrix.com/trac/
- nginx / gunicorn for your python web server stack
http://www.nginx.com/
http://gunicorn.org/Re: Python Deployment Anti-Patterns
#5> The trick is to build a debian package (but it can be done using RPMs just as well) with the application and the whole virtualenv inside.
I would love to read an article describing some best practices for doing that. I tried it once and found it extremely difficult, reverting to a git checkout + virtualenv kind of deployment.
Re: Python Deployment Anti-Patterns
#6Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist. Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment proces…
> [...] you can knock out two birds with one stone by just using pythonbrew
Re: Python Deployment Anti-Patterns
#7Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist. Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment proces…
> Those tasks are better suited for Fabric's imperative design. Yes they are, but IMHO not on the target servers. I use Fabric to build DEBs that get deployed by Puppet. I prefer to have no build tools on target servers, YMMV.
DEBs for your own project files, or debs containing built python modules that the server needs to run those project files?
If the latter, how well does that mix in with virtualenv? or do you just avoid it entirely?
Re: Python Deployment Anti-Patterns
#8As a python dev who deploys a lot of software, I found this article to be wonderfully helpful and informative, and a good reflection of current best practices. Summary of the deployment tools mentioned: - Manage remote daemons with supervisord http://supervisord.org/ - Manage python packages with pip (and use `pip freeze`) http://pypi.python.org/pypi/pip http://www.pip-installer.org/en/latest/requirements.html - Mana…
Yes to all of this and I'll throw in my two cents on our web server setup which is nginx + uwsgi (which has served remarkably well).
Re: Python Deployment Anti-Patterns
#9Earlier quoted context omitted.
> Those tasks are better suited for Fabric's imperative design. Yes they are, but IMHO not on the target servers. I use Fabric to build DEBs that get deployed by Puppet. I prefer to have no build tools on target servers, YMMV.
DEBs for your own project files, or debs containing built python modules that the server needs to run those project files? If the latter, how well does that mix in with virtualenv? or do you just avoid it entirely?
I’m the dude that wrote article, so like it says: Own stuff + deps.
Your can re-initialize a virtualenv to fix it by simply running virtualenv again.
But pinky swear I’ll write the second article. ;)
Re: Python Deployment Anti-Patterns
#10> The trick is to build a debian package (but it can be done using RPMs just as well) with the application and the whole virtualenv inside. I would love to read an article describing some best practices for doing that. I tried it once and found it extremely difficult, reverting to a git checkout + virtualenv kind of deployment.
It will come, I promise.