Python Deployment Anti-Patterns • Hacker News Board

Re: Python Deployment Anti-Patterns

#71

Post by njharman » Tue, Apr 24, 2012, 6:39 AM UTC

j_baker wrote: post #37

viraptor wrote: post #28

Earlier quoted context omitted.

> "into global site-packages that you shouldn’t use for any serious coding" Any specific reason for that? I find it quite good and have quite large deployments using .debs only with packages in global location. (tens of packages produced locally - either updated or unavailable dependencies and the service itself) Any direct dependency is handled by package pinning and no update goes into production untested, so the w…

Having a global python installation where packages are constantly installed, uninstalled, and updated is the path to madness. If something goes wrong, what can you do? You can't wipe out the system python. You can wipe out a virtualenv though.

> what can you do?

Understand what went wrong and not ignorantly hit Ctrl-Alt-Delete.

Re: Python Deployment Anti-Patterns

#72

Post by njharman » Tue, Apr 24, 2012, 6:43 AM UTC

jordanb wrote: post #20

This guy seems to be of the opinion that the software should be completely isolated from the deployment operating system. I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the lin…

I draw the line at system libraries and long established system services email, cron, syslog, dns, etc.

I always want to compile my full stack; DB, network servers, language.

Re: Python Deployment Anti-Patterns

#73

Post by njharman » Tue, Apr 24, 2012, 6:43 AM UTC

jordanb wrote: post #20

This guy seems to be of the opinion that the software should be completely isolated from the deployment operating system. I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the lin…

I draw the line at system libraries and long established system services email, cron, syslog, dns, etc.

I always want to compile my full stack; DB, network servers, language.

Re: Python Deployment Anti-Patterns

#74

Post by mh- » Tue, Apr 24, 2012, 6:49 AM UTC

njharman wrote: post #69

Virtualenv is a half solution and a hack. Use vagrant and VMs. There's a whole sea of libs and software that isn't "versioned" by pip/virtualenv. supervisord is the wrong solution. It answers the wrong question (is the process running). It's worse than useless in that it has given false positives. The right question is (is the process responding correctly). Use monit or something else that actually does what's needed…

virtualenv isn't just used for testing, though. it's widely used to isolate applications' python environments/dependencies from one another.

Re: Python Deployment Anti-Patterns

#75

Post by spahl » Tue, Apr 24, 2012, 7:12 AM UTC

edanm wrote: post #70

alexatkeplar wrote: post #57

Earlier quoted context omitted.

Likewise +1 to Nginx + uwsgi. In the various performance benchmarks* uwsgi beats gunicorn, and uwsgi comes bundled with Nginx now, so there's really no reason to bother with setting up gunicorn any more. * http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi http://nichol.as/benchmark-of-python-web-servers

uwsgi comes bundles with Nginx? First I've heard of that. Very interesting. Do you have any easy tutorials on getting Nginx + uwsgi set up?

The uwsgi upstream module has been bundled for a long time now.

Here is the doc from the uwsgi site: http://projects.unbit.it/uwsgi/wiki/RunOnNginx

Re: Python Deployment Anti-Patterns

#76

Post by antihero » Tue, Apr 24, 2012, 7:28 AM UTC

viraptor wrote: post #28

hynek wrote: post #17

Earlier quoted context omitted.

The worst thing about them is the fact that they are installed into global site-packages that you shouldn’t use for any serious coding. And yes, they are mostly outdated too.

> "into global site-packages that you shouldn’t use for any serious coding" Any specific reason for that? I find it quite good and have quite large deployments using .debs only with packages in global location. (tens of packages produced locally - either updated or unavailable dependencies and the service itself) Any direct dependency is handled by package pinning and no update goes into production untested, so the w…

Your system python applications might have different requirements to your project. Just use virtualenv, it's easier and less error-prone.

Re: Python Deployment Anti-Patterns

#77

Post by antihero » Tue, Apr 24, 2012, 7:29 AM UTC

hynek wrote: post #53

stephen_mcd wrote: post #52

Earlier quoted context omitted.

Fabric seems to be the most popular deployment tool, yet the author advises against it without giving any reason why. I'd love to see some proper detail in the article around why. And not in the vein of "Chef/Puppet are better", but more along the line of "here's what can go wrong with Fabric".

Fabric is an invaluable deployment tool, but you have to know its boundaries for what it was intended. With Fabric you tell what to _do_ and with Puppet/Chef you define what the result should _look_ like. You define how a server should look like and it can make sure its true for 1000 servers. Or 10000. It’s not about Fabric vs. Puppet, but how to use both in the most efficient way.

It would be fantastic if someone were to write a tutorial how to deploy, for instance, a simple Flask+MySQL or Django+PostgreSQL application using these tools. I'm at a loss as to where to start.

Re: Python Deployment Anti-Patterns

#78

Post by antihero » Tue, Apr 24, 2012, 7:32 AM UTC

njharman wrote: post #69

Virtualenv is a half solution and a hack. Use vagrant and VMs. There's a whole sea of libs and software that isn't "versioned" by pip/virtualenv. supervisord is the wrong solution. It answers the wrong question (is the process running). It's worse than useless in that it has given false positives. The right question is (is the process responding correctly). Use monit or something else that actually does what's needed…

Why would you want to virtualise an entire system when your dependencies are restricted to a bunch of Python modules?

Re: Python Deployment Anti-Patterns

#79

Post by antihero » Tue, Apr 24, 2012, 7:41 AM UTC

jordanb wrote: post #20

This guy seems to be of the opinion that the software should be completely isolated from the deployment operating system. I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the lin…

> isolated

Perhaps a better term would be "decoupled"?

Re: Python Deployment Anti-Patterns

#80

Post by cageface » Tue, Apr 24, 2012, 8:36 AM UTC

The ability of Go to produce a single, self-contained executable is one of the biggest advantages it has over the "scripting" languages. It makes deployment so much simpler.