Live data from Hacker News

Python Deployment Anti-Patterns

hynek.me

11–20 of 125 posts

Re: Python Deployment Anti-Patterns

#11
We do this, but also add a couple layers of safety between us and PyPI:

1. Run your own secure, local pypi clone with exact source versions of the packages you use.

2. The packages for production are built into RPMs from the local pypi.

PyPI is great for discovery, getting things running quickly, and testing new versions, but you never want to rely on it, even for development.

Re: Python Deployment Anti-Patterns

#12

We do this, but also add a couple layers of safety between us and PyPI: 1. Run your own secure, local pypi clone with exact source versions of the packages you use. 2. The packages for production are built into RPMs from the local pypi. PyPI is great for discovery, getting things running quickly, and testing new versions, but you never want to rely on it, even for development.

Yeah, plus it speeds package building up and protects against not-so-occasional pypi outages.

Re: Python Deployment Anti-Patterns

#13
I don't think using virtualenv to jam everything into a big deb file is really a best practice.

But at the end of the day, I do have to do a lot of that with application deployment, but I try to only go as far as packaging libraries (ie. gems, jars, python equiv) in the rpm/deb file.

RHEL 6 is python 2.6.6, btw.

What happens when there are vulns for your stack?

Re: Python Deployment Anti-Patterns

#14
I don't get the negativity on using your distro's packages, at least from the staying-stable perspective. Any decent package manager should let you pin/hold critical packages on a particular version, so if "the next Ubuntu ships with a different SQLAlchemy by default" you just hold the SQLAlchemy package at the version you want and then ignore it until you're ready to make that move.

99% percent of the time when I hear people complaining about their distro's packages, the complaints are coming from the opposite direction -- they want to run something bleeding-edge and the distro doesn't have it yet. (This is the standard beef Rubyists have with Debian, for instance -- that code that just hit Github ten minutes ago isn't in Debian's repos yet.)

Re: Python Deployment Anti-Patterns

#15

I don't get the negativity on using your distro's packages, at least from the staying-stable perspective. Any decent package manager should let you pin/hold critical packages on a particular version, so if "the next Ubuntu ships with a different SQLAlchemy by default" you just hold the SQLAlchemy package at the version you want and then ignore it until you're ready to make that move. 99% percent of the time when I he…

They rarely have a modern enough package to be worthwhile. When you control the package pipeline, you have the ability to dictate how modern or conservative your software is.

The problem is that most programming languages (especially Python and Ruby) are ecosystems unto themselves and often move at a much faster pace than any stable distro (or LTS) could keep up with. That's why we have gems and pip.

Cf. Ubuntu LTS MongoDB default is like 1.2 or something. This is why I switched to using 10gen's repo.

Re: Python Deployment Anti-Patterns

#16

I don't think using virtualenv to jam everything into a big deb file is really a best practice. But at the end of the day, I do have to do a lot of that with application deployment, but I try to only go as far as packaging libraries (ie. gems, jars, python equiv) in the rpm/deb file. RHEL 6 is python 2.6.6, btw. What happens when there are vulns for your stack?

> What happens when there are vulns for your stack?

That’s a good point and the answer is: You have to monitor your dependencies of public services (that aren’t that many).

But you have to do that anyway, because I can’t explain to our customers that their data has been hacked because Ubuntu/Red Hat didn’t update Django (fast enough).

Re: Python Deployment Anti-Patterns

#17

I don't get the negativity on using your distro's packages, at least from the staying-stable perspective. Any decent package manager should let you pin/hold critical packages on a particular version, so if "the next Ubuntu ships with a different SQLAlchemy by default" you just hold the SQLAlchemy package at the version you want and then ignore it until you're ready to make that move. 99% percent of the time when I he…

The worst thing about them is the fact that they are installed into global site-packages that you shouldn’t use for any serious coding.

And yes, they are mostly outdated too.

Re: Python Deployment Anti-Patterns

#18

I don't get the negativity on using your distro's packages, at least from the staying-stable perspective. Any decent package manager should let you pin/hold critical packages on a particular version, so if "the next Ubuntu ships with a different SQLAlchemy by default" you just hold the SQLAlchemy package at the version you want and then ignore it until you're ready to make that move. 99% percent of the time when I he…

They rarely have a modern enough package to be worthwhile. When you control the package pipeline, you have the ability to dictate how modern or conservative your software is. The problem is that most programming languages (especially Python and Ruby) are ecosystems unto themselves and often move at a much faster pace than any stable distro (or LTS) could keep up with. That's why we have gems and pip. Cf. Ubuntu LTS M…

Perhaps, but that's a completely different argument than the one TFA is making.

Besides, in the Ubuntu case anyway, LTS is supposed to be old. The whole point of LTS releases is to let slow-moving institutions/enterprises sit on ancient packages for 3-5 years without having to worry about backporting security updates. If you want recent versions of packages LTS is precisely the wrong place for you to be.

Re: Python Deployment Anti-Patterns

#19

Earlier quoted context omitted.

They rarely have a modern enough package to be worthwhile. When you control the package pipeline, you have the ability to dictate how modern or conservative your software is. The problem is that most programming languages (especially Python and Ruby) are ecosystems unto themselves and often move at a much faster pace than any stable distro (or LTS) could keep up with. That's why we have gems and pip. Cf. Ubuntu LTS M…

Perhaps, but that's a completely different argument than the one TFA is making. Besides, in the Ubuntu case anyway, LTS is supposed to be old. The whole point of LTS releases is to let slow-moving institutions/enterprises sit on ancient packages for 3-5 years without having to worry about backporting security updates. If you want recent versions of packages LTS is precisely the wrong place for you to be.

LTS is the default distro for a lot of server and VPS providers. Often the latest VM images will be rather...unstable, even putting aside the instability of the distro itself.

Re: Python Deployment Anti-Patterns

#20
This guy seems to be of the opinion that the software should be completely isolated from the deployment operating system.

I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the line? He seems to draw it further down the stack than most people (no system python, for instance) but he doesn't give his opinion on, for instance, using the system postgresql.

Anyway, I personally would draw the line further up the stack than him, but take things on a case-by-case basis, and I don't really consider it an "anti-pattern."

With regards to fabric vs. puppet, I understand the advantages of puppet when you have a complicated, hetrogenous deployment environment. But the majority of projects I've worked on have the operations model of a set of identically-configured application servers back-ended against a database server. For this configuration, what does puppet give you? If the author's argument is that the site may eventually outgrow that model, well, I can see puppet becoming necessary, but why not cross that bridge when you get to it?

Post reply on HN