Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python Deployment Anti-Patterns

hynek.me

Post reply on HN
Print view View original

101–110 of 125 posts

Re: Python Deployment Anti-Patterns

#101

Post by read_wharf »

read_wharf wrote: post #97
IgorPartola wrote: post #96

My two cents: I am a developer + ops person and deploy Python apps all the time. Typically they are Django and Tornado services. On top we also have a lot of daemons and a ton of library code. I agree with the OP on most points but do not on a few. First DO use packages that come with the OS. The OP says that you should not have the distro maintainers dictating what you use. I say, use what is widely available. It ta…

"First DO use packages that come with the OS." I'd go middle ground, and start here, but consider a self-built package where necessary. It depends in part on the focus of your distro. virtualenv. What problem does it solve? Different python version/environments? Wouldn't that be better solved with another (virtual) server? I understand if an extra $20/month is an issue, but otherwise ...

I have just been educated by reading more of this thread. I can see an obvious need for one virtualenv, so that you can separate your service and its needs from the system python and its needs. Beyond that my inclination would be to go more servers rather than more virtualenvs, but circumstances vary and my experience is narrow.

Re: Python Deployment Anti-Patterns

#102

Post by hynek »

read_wharf wrote: post #101
read_wharf wrote: post #97

Earlier quoted context omitted.

"First DO use packages that come with the OS." I'd go middle ground, and start here, but consider a self-built package where necessary. It depends in part on the focus of your distro. virtualenv. What problem does it solve? Different python version/environments? Wouldn't that be better solved with another (virtual) server? I understand if an extra $20/month is an issue, but otherwise ...

I have just been educated by reading more of this thread. I can see an obvious need for one virtualenv, so that you can separate your service and its needs from the system python and its needs. Beyond that my inclination would be to go more servers rather than more virtualenvs, but circumstances vary and my experience is narrow.

Not every Python application is a big web app. We have systems that run several smaller Python apps. Python is everywhere.

Re: Python Deployment Anti-Patterns

#103

Post by IgorPartola »

hynek wrote: post #98
IgorPartola wrote: post #96

My two cents: I am a developer + ops person and deploy Python apps all the time. Typically they are Django and Tornado services. On top we also have a lot of daemons and a ton of library code. I agree with the OP on most points but do not on a few. First DO use packages that come with the OS. The OP says that you should not have the distro maintainers dictating what you use. I say, use what is widely available. It ta…

> First DO use packages that come with the OS. The OP says that you should not have the distro maintainers dictating what you use. I say, use what is widely available. It takes the headache out of a lot of your deployments. If you are looking for a library that converts foo to bar look in your distro's repos before going on GitHub. Your sysadmin will thank you. The sysadmin will have no part in the game if you use pa…

Sysadmin/Ops should have a part in what gets deployed. It is their job. My point is that if you have a choice between Python 2.6 that comes with your distro and Python 2.7 that doesn't, go with 2.6. The cost is minimal. Same for various libraries (PIL, NumPy, etc.) I currently have to deploy Django-Piston directly from a master branch on GitHub due to some terrible decisions made by developers who never gave deployment a second thought. Avoid this. If you would save more hours by using a newer library, minus time it takes to set it up and maintain it in production, go for it, but don't discount the cost on the Ops side.

I don't have a problem with virtualenv: it's a fine development tool, but it is not what I would use in production. If you want separate clean environments for each app, use KVM or Xen and give it a whole server.

supervisord is a fine solution. I just prefer that my processes look exactly like system processes. Upstart and rc.d are fantastic and do everything I need well.

As for separation of code and configuration, I simply mean that in my case I use Debian packages to deploy all of our software. This means that each package must be generic enough that it is deployable so long as its dependencies are satisfied. Thus your config files are mostly external to your packages. Then you can easily use Puppet or some such to deploy code. One other reason to use native distro packages: Puppet does not play well with pip/easy_install, etc.

>> Lastly, DO use apache + mod_wsgi. It is fast, stable, widely supported and well tested.

> And nginx + uwsgi/gunicorn aren’t?

I didn't say that. I am simply stating that saying "don't use apache" is wrong. Do use it. You can also use nginx + uwsgi/gunicorn if you want to, but apache is by no means a bad choice. It's got 25 years of use and nobody that uses it seriously is complaining.

> Absolutely. nginx is way past the “new and hacky” state though.

It is not, and I never said so. I use nginx + apache, where nginx is a reverse proxy. In fact nginx is my top choice for front-end server setups. I am saying, don't deploy things directly out of GitHub. Go with slightly older, more tested stuff. It'll be a bigger payoff in the end.

Overall, I think we are saying the same thing, with slightly different tools we normally reach for. I am just trying to throw a different perspective out there and a different way to do things. Thanks for the detailed reply.

Re: Python Deployment Anti-Patterns

#104

Post by hynek »

IgorPartola wrote: post #103
hynek wrote: post #98

Earlier quoted context omitted.

> First DO use packages that come with the OS. The OP says that you should not have the distro maintainers dictating what you use. I say, use what is widely available. It takes the headache out of a lot of your deployments. If you are looking for a library that converts foo to bar look in your distro's repos before going on GitHub. Your sysadmin will thank you. The sysadmin will have no part in the game if you use pa…

Sysadmin/Ops should have a part in what gets deployed. It is their job. My point is that if you have a choice between Python 2.6 that comes with your distro and Python 2.7 that doesn't, go with 2.6. The cost is minimal. Same for various libraries (PIL, NumPy, etc.) I currently have to deploy Django-Piston directly from a master branch on GitHub due to some terrible decisions made by developers who never gave deployme…

> I currently have to deploy Django-Piston directly from a master branch on GitHub due to some terrible decisions made by developers who never gave deployment a second thought. Avoid this.

JFTR, there is a pretty good solution for this (because in the real world, you can’t always avoid that): a custom pypi server.

You take a git version you know that works do a `python setup.py sdist` und push it to a private repo. We have to do stuff like this for Sybase drivers for example which are open source but not in PyPI (or any distribution). It saves so much pain.

> As for separation of code and configuration, I simply mean that in my case I use Debian packages to deploy all of our software.

Well we do the same but the Debian packages contain code _and_ the virtualenv.

> Overall, I think we are saying the same thing, with slightly different tools we normally reach for.

Mostly yes. Just wanted to add the pypi tip as it hasn’t been mentioned yet in this thread.

Re: Python Deployment Anti-Patterns

#105

Post by dignan »

alexatkeplar wrote: post #57
Ixiaus wrote: post #8

Earlier quoted context omitted.

Yes to all of this and I'll throw in my two cents on our web server setup which is nginx + uwsgi (which has served remarkably well).

Likewise +1 to Nginx + uwsgi. In the various performance benchmarks* uwsgi beats gunicorn, and uwsgi comes bundled with Nginx now, so there's really no reason to bother with setting up gunicorn any more. * http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi http://nichol.as/benchmark-of-python-web-servers

Can you restart the uwsgi instance(s) without restarting nginx? That's a pretty significant benefit of gunicorn...

Re: Python Deployment Anti-Patterns

#106

Post by danssig »

cdavid wrote: post #68

I am glad this is working for the OP, but pushing virtualenv and "self-contained" apps as the one solution is a diservice to the community. There are valid reasons to rely on your OS, assuming you have an homogenous deployment target (same OS, maybe different versions): - lots of people argue for virtualenv because some versions may be incompatible. The problem here is the lack of backward compatibility of packages,…

I disagree. IMO dynamic load libraries was a known bad solution before they were made (dll hell). Plenty of exploits have come from them, plenty of horror stories, etc. If you have a nice distro anyway, upgrading a given library that everyone packages should be doable.

Windows still uses shared libraries but at least they invented the GAC so applications can specify exactly which version of a library they work with and the installer will install that version if it's not present.

Re: Python Deployment Anti-Patterns

#107

Post by ch0wn »

davvid wrote: post #89
ch0wn wrote: post #5

> The trick is to build a debian package (but it can be done using RPMs just as well) with the application and the whole virtualenv inside. I would love to read an article describing some best practices for doing that. I tried it once and found it extremely difficult, reverting to a git checkout + virtualenv kind of deployment.

Check out git's "make rpm" target. https://github.com/gitster/git/blob/master/Makefile Hosting your own apt/yum repo is pretty simple. Does anyone have an example of a similar "make deb" target they could share? I've heard of git-dpm and git-buildpackage but haven't used them extensively myself. They're the debian git packaging tools. http://wiki.debian.org/PackagingWithGit

I'm familiar with building and hosting deps in general. The problem I had was that I tried to build debs for every single dependency which turned out to be really hard to automate. That's why I'm really interested in seeing an approach with the virtualenv embedded into the package.

Re: Python Deployment Anti-Patterns

#108

Post by serverascode »

hynek wrote: post #16
serverascode wrote: post #13

I don't think using virtualenv to jam everything into a big deb file is really a best practice. But at the end of the day, I do have to do a lot of that with application deployment, but I try to only go as far as packaging libraries (ie. gems, jars, python equiv) in the rpm/deb file. RHEL 6 is python 2.6.6, btw. What happens when there are vulns for your stack?

> What happens when there are vulns for your stack? That’s a good point and the answer is: You have to monitor your dependencies of public services (that aren’t that many). But you have to do that anyway, because I can’t explain to our customers that their data has been hacked because Ubuntu/Red Hat didn’t update Django (fast enough).

You make it sound like if you do one then you can do 100. Not the case.

Re: Python Deployment Anti-Patterns

#109

Post by hynek »

serverascode wrote: post #108
hynek wrote: post #16

Earlier quoted context omitted.

> What happens when there are vulns for your stack? That’s a good point and the answer is: You have to monitor your dependencies of public services (that aren’t that many). But you have to do that anyway, because I can’t explain to our customers that their data has been hacked because Ubuntu/Red Hat didn’t update Django (fast enough).

You make it sound like if you do one then you can do 100. Not the case.

My public services don’t have 100 dependencies and that’s on purpose. Relying on magic distribution fairies for all your libraries is a IMHO a false sense of security, YMMV.

How do you make sure that whenever one of your dependencies gets updated that your daemons get restarted?

And what do you do if you need a package that isn’t part of your distribution?

Re: Python Deployment Anti-Patterns

#110

Post by pbiggar »

davvid wrote: post #91
pbiggar wrote: post #60

Earlier quoted context omitted.

Well, not really, because what if ubuntu packages rely on version X, and you need version Y.

You roll version Y yourself and install it into an alternate prefix. If the server uses debian that means make a new deb and deploy it using the standard tools. apt-get/yum/etc. are very solid deployment tools.

I wasn't saying it was impossible, but what you've described is already about 10 times harder than using virtualenv.
Post reply on HN