Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Deploy Python Code

nylas.com

Post reply on HN
Print view View original

21–30 of 129 posts

Re: How We Deploy Python Code

#21

Post by emidln »

kbar13 wrote: post #2

http://pythonwheels.com/ solves the problem of building c extensions on installation.

Pair this with virtualenvs in separate directories (so that "rollback" is just a ssh mv and a reload for whatever supervisor process) and you get to skip the mess of building system packages.

Also, are there seriously places that don't run their own PyPI mirrors? Places that have people who understand how to integrate platform-specific packages but can't be bothered to deploy one of the several PyPI-in-a-box systems or pay for a hosted PyPI?

Re: How We Deploy Python Code

#22

Post by knicholes »

craigmccaskill wrote: post #11
sandGorgon wrote: post #5

The fact that we had a weird combination of python and libraries took us towards Docker. And we have never looked back. For someone trying out building python deployment packages using deb, rpm, etc. I really recommend Docker.

They specifically called that out in the article with an entire section called "just use docker".

On the same vein as omni, the other reasons are "we don't want to learn a new technology, even though it was made exactly for this purpose" and "we don't want to learn how to install docker-registry."

Re: How We Deploy Python Code

#24

Post by craigmccaskill »

omni wrote: post #19
jlees wrote: post #15

Earlier quoted context omitted.

To be fair, what didn't work for Nylas might well not be an issue for others. There's definitely more than one way to skin a cat, especially in the Python world.

Seriously, their #1 reason for not using Docker amounts to "we couldn't be bothered to update our kernel." This reads to me more of an interesting story about how they chose to solve the problem rather than a serious recommendation of how things should be done.

I don't think that's a fair representation of why they don't want to update kernels and also completely downplays that there were three other points mentioned which all added non trivial overhead to getting stuff done now.

In an ideal world you build a system from the ground up, but rarely is that ever possible and the approach taken to iterate is far more valuable than your requested 'serious' recommendation.

Re: How We Deploy Python Code

#25

Post by hcrisp »

TDL wrote: post #12
ah- wrote: post #4

conda works pretty well.

Agreed (although biased since I used to work at Continuum.) I am wondering what others think of conda?

I use conda as well and found it to be great. I love how it detects and manages dependencies for you when you install a new module.

Re: How We Deploy Python Code

#26

Post by x0x0 »

craigmccaskill wrote: post #11
sandGorgon wrote: post #5

The fact that we had a weird combination of python and libraries took us towards Docker. And we have never looked back. For someone trying out building python deployment packages using deb, rpm, etc. I really recommend Docker.

They specifically called that out in the article with an entire section called "just use docker".

if you have to choose between a kernel and docker, just choose docker. Python can't get their shit together deployment-wise, and docker is the one true route (tm) to python deployment happiness.

forget virtualenv; forget package dependencies on conflicting versions of libxml; forget coworkers that have 3 different conflicting versions of requests scattered through various services, and goddamnit I just want to run a dev build; forget coworkers that scribble droppings all over the filesystem, and assume certain services will never coexist on the same box

just use docker. It's going to go like this:

step 1: docker

step 2: happy

Re: How We Deploy Python Code

#27

Post by toomuchtodo »

emidln wrote: post #21
kbar13 wrote: post #2

http://pythonwheels.com/ solves the problem of building c extensions on installation.

Pair this with virtualenvs in separate directories (so that "rollback" is just a ssh mv and a reload for whatever supervisor process) and you get to skip the mess of building system packages. Also, are there seriously places that don't run their own PyPI mirrors? Places that have people who understand how to integrate platform-specific packages but can't be bothered to deploy one of the several PyPI-in-a-box systems…

> Also, are there seriously places that don't run their own PyPI mirrors? Places that have people who understand how to integrate platform-specific packages but can't be bothered to deploy one of the several PyPI-in-a-box systems or pay for a hosted PyPI?

Yes. I've seen them, and they've been huge shops.

Re: How We Deploy Python Code

#28

Post by x0x0 »

jedberg wrote: post #8
sandGorgon wrote: post #5

The fact that we had a weird combination of python and libraries took us towards Docker. And we have never looked back. For someone trying out building python deployment packages using deb, rpm, etc. I really recommend Docker.

There was a whole paragraph in the article about why Docker didn't work for them.

they had 3 reasons:

one of which was just silly (kernel version -- are you living on that point release forever?)

one of which was valid (necessity to maintain method for distributing docker images), but probably dumb: you only get so many innovation points per company, and innovating on a problem docker just solves means you are supporting your in-house solution ad infinitum

and one of which definitely sounds painful (docker vs extant ansible playbooks)

Re: How We Deploy Python Code

#29

Post by tschellenbach »

Yes, someone should build the one way to ship your app. No reason for everybody to be inventing this stuff over and over again.

Deploys are harder if you have a large codebase to ship. rSync works really well in those cases. It requires a bit of extra infrastructure, but is super fast.

Re: How We Deploy Python Code

#30

Post by vamega »

emidln wrote: post #21
kbar13 wrote: post #2

http://pythonwheels.com/ solves the problem of building c extensions on installation.

Pair this with virtualenvs in separate directories (so that "rollback" is just a ssh mv and a reload for whatever supervisor process) and you get to skip the mess of building system packages. Also, are there seriously places that don't run their own PyPI mirrors? Places that have people who understand how to integrate platform-specific packages but can't be bothered to deploy one of the several PyPI-in-a-box systems…

Can you point me to your recommended PyPI-in-a-box system?
Post reply on HN