Regarding virtualenv, I have come to the conclusion that Linux containers are robust enough now (like freebsd jails say two or three years ago) that I don't need to virtualise just python - I can afford to have the whole server as a "virtualenv" - no need for that extra complexity just install into site packages. No conflicts because a whole instance is dedicated. Jails take this to the limit - one virtual machine, one process - say Bind. A vulnerability in Bind ? The attacker takes over ... Nothing.
Python Deployment Anti-Patterns
51–60 of 125 posts
Re: Python Deployment Anti-Patterns
#52Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist. Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment proces…
Fabric seems to be the most popular deployment tool, yet the author advises against it without giving any reason why.
I'd love to see some proper detail in the article around why.
And not in the vein of "Chef/Puppet are better", but more along the line of "here's what can go wrong with Fabric".
Re: Python Deployment Anti-Patterns
#53Regarding "Don't use ancient system Python versions" and "Use virtual environments", you can knock out two birds with one stone by just using pythonbrew. It also saves you the hassle of rolling your own deb/rpm if a package doesn't happen to exist. Also, Chef/Puppet aren't "alternatives" to something like Fabric. Use the former for server provisioning, and use the latter for actually kicking off the deployment proces…
Fabric seems to be the most popular deployment tool, yet the author advises against it without giving any reason why. I'd love to see some proper detail in the article around why. And not in the vein of "Chef/Puppet are better", but more along the line of "here's what can go wrong with Fabric".
Fabric is an invaluable deployment tool, but you have to know its boundaries for what it was intended.
With Fabric you tell what to _do_ and with Puppet/Chef you define what the result should _look_ like.
You define how a server should look like and it can make sure its true for 1000 servers. Or 10000.
It’s not about Fabric vs. Puppet, but how to use both in the most efficient way.
Re: Python Deployment Anti-Patterns
#54Earlier quoted context omitted.
I think there's an exageration as well, today the trend seems to be "use nothing by the distro" Ok, sure, MongoDB still changes a lot between versions, in this case you should use the latest version. But stop there. Especially if you're paying for support (like RHEL) There should be a good reason for you to compile Apache / MySQL / PostgreSQL / Python. Otherwise, use the distro version. One (common) exception would b…
Yes you should, but if you have them: do it. That’s what the article says. ;) Please don’t push it in a wrong direction. What I actually wrote is: because we’re a LAMP web hoster, we compile MySQL+PHP+Apache ourself. And because we’re a Python shop, we don’t let Ubuntu/Red Hat dictate which Python version we use.
You're right, sorry about that ;) Guess I was thinking too much about the "compile everything" people.
Great article, btw!
Re: Python Deployment Anti-Patterns
#55Earlier quoted context omitted.
Perhaps, but that's a completely different argument than the one TFA is making. Besides, in the Ubuntu case anyway, LTS is supposed to be old. The whole point of LTS releases is to let slow-moving institutions/enterprises sit on ancient packages for 3-5 years without having to worry about backporting security updates. If you want recent versions of packages LTS is precisely the wrong place for you to be.
I would disagree with this. LTS (I'm referring to Ubuntu LTS as well as RHEL) is still a good fit because there are 9001 other packages your system relies on for day-to-day operations, and I'd rather mess with those as least often as possible. If you don't use a LTS then you'll have to do whole system upgrades every few years - or else you stop getting security backports. I'd much rather micromanage the dozen highly…
there's still a chance that you'll be stuck with the just-before-latest-stable version.
Is that so bad? Does python really change that much from release to release?
Re: Python Deployment Anti-Patterns
#56Earlier quoted context omitted.
Yes you should, but if you have them: do it. That’s what the article says. ;) Please don’t push it in a wrong direction. What I actually wrote is: because we’re a LAMP web hoster, we compile MySQL+PHP+Apache ourself. And because we’re a Python shop, we don’t let Ubuntu/Red Hat dictate which Python version we use.
You're right, sorry about that ;) Guess I was thinking too much about the "compile everything" people. Great article, btw!
thanks!
Re: Python Deployment Anti-Patterns
#57As a python dev who deploys a lot of software, I found this article to be wonderfully helpful and informative, and a good reflection of current best practices. Summary of the deployment tools mentioned: - Manage remote daemons with supervisord http://supervisord.org/ - Manage python packages with pip (and use `pip freeze`) http://pypi.python.org/pypi/pip http://www.pip-installer.org/en/latest/requirements.html - Mana…
Yes to all of this and I'll throw in my two cents on our web server setup which is nginx + uwsgi (which has served remarkably well).
Likewise +1 to Nginx + uwsgi. In the various performance benchmarks* uwsgi beats gunicorn, and uwsgi comes bundled with Nginx now, so there's really no reason to bother with setting up gunicorn any more.
* http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi http://nichol.as/benchmark-of-python-web-servers
Re: Python Deployment Anti-Patterns
#58Earlier quoted context omitted.
DEBs for your own project files, or debs containing built python modules that the server needs to run those project files? If the latter, how well does that mix in with virtualenv? or do you just avoid it entirely?
I’m the dude that wrote article, so like it says: Own stuff + deps. Your can re-initialize a virtualenv to fix it by simply running virtualenv again. But pinky swear I’ll write the second article. ;)
Get on it! :) I really enjoyed the article and want to know more of your magic.
Re: Python Deployment Anti-Patterns
#59Regarding virtualenv, I have come to the conclusion that Linux containers are robust enough now (like freebsd jails say two or three years ago) that I don't need to virtualise just python - I can afford to have the whole server as a "virtualenv" - no need for that extra complexity just install into site packages. No conflicts because a whole instance is dedicated. Jails take this to the limit - one virtual machine, o…
Sorry, but I'm not sure I understand your logic. Using virtualenv adds extra complexity, but virtualizing the entire server doesn't? I mean, the only complexity using virtualenv adds is having to run the virtualenv process once. After that, you can still install to site-packages. You just have to install to a different site-packages directory.
Besides that, it's worth pointing out that using a virtualenv is not a security precaution. It's a precaution to prevent mucking up the global python installation for other packages that run on it. Using linux containers to achieve this seems like overkill.
Re: Python Deployment Anti-Patterns
#60Earlier quoted context omitted.
Mainly it's about incompatibilities. What if you have two apps that require different versions of a library? If you've installed it in site-packages, then you have little recourse. By separating them out with virtualenv the two apps will work just fine.
Fortunately I'm in a one-service-one-server environment, so I may be biased here ;)
Well, not really, because what if ubuntu packages rely on version X, and you need version Y.