Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Deploy Python Code

nylas.com

Post reply on HN
Print view View original

11–20 of 129 posts

Re: How We Deploy Python Code

#11

Post by craigmccaskill »

sandGorgon wrote: post #5

The fact that we had a weird combination of python and libraries took us towards Docker. And we have never looked back. For someone trying out building python deployment packages using deb, rpm, etc. I really recommend Docker.

They specifically called that out in the article with an entire section called "just use docker".

Re: How We Deploy Python Code

#13

Post by remh »

We fixed that issue at Datadog by using Chef Omnibus:

https://www.datadoghq.com/blog/new-datadog-agent-omnibus-tic...

It's more complicated than the proposed solution by nylas but ultimately it gives you full control of the whole environment and ensure that you won't hit ANY dependency issue when shipping your code to weird systems.

Re: How We Deploy Python Code

#15

Post by jlees »

craigmccaskill wrote: post #11
sandGorgon wrote: post #5

The fact that we had a weird combination of python and libraries took us towards Docker. And we have never looked back. For someone trying out building python deployment packages using deb, rpm, etc. I really recommend Docker.

They specifically called that out in the article with an entire section called "just use docker".

To be fair, what didn't work for Nylas might well not be an issue for others. There's definitely more than one way to skin a cat, especially in the Python world.

Re: How We Deploy Python Code

#16

Post by 4lejandrito »

TDL wrote: post #12
ah- wrote: post #4

conda works pretty well.

Agreed (although biased since I used to work at Continuum.) I am wondering what others think of conda?

We have used Conda for our first python deployment and the process has been seamless. It provides the same sandboxing concept using virtualenvs and also uses prebuilt binaries for native dependencies so you don't have to build them every time. The only drawback I would say is that we have to install miniconda in our production servers rather than just deploying an standalone package.

Re: How We Deploy Python Code

#17

Post by rhelmer »

jlees wrote: post #10
Cieplak wrote: post #3

Highly recommend FPM for creating packages (deb, rpm, osx .pkg, tar) from gems, python modules, and pears. https://github.com/jordansissel/fpm

That seems like a neat tool. I wonder if you could combine it with the sandboxing that dh-virtualenv provides to get the best of both worlds?

I've used fpm to make rpm and deb packages that simply include a virtualenv, it works ok.

One of the significant tradeoffs to this approach is you lose the carefully-crafted tree-of-dependencies that the distros favor, so it makes the package pretty much automatically unacceptable to package maintainers.

However, being able to have install instructions that amount to "yum/apt-get install " is pretty great.

I am hoping for an app/container convergence at some point, but we might need to drop the fine-grained dependency dream and have them be more self-contained, like Mac OS X apps.

Re: How We Deploy Python Code

#19

Post by omni »

jlees wrote: post #15
craigmccaskill wrote: post #11

Earlier quoted context omitted.

They specifically called that out in the article with an entire section called "just use docker".

To be fair, what didn't work for Nylas might well not be an issue for others. There's definitely more than one way to skin a cat, especially in the Python world.

Seriously, their #1 reason for not using Docker amounts to "we couldn't be bothered to update our kernel." This reads to me more of an interesting story about how they chose to solve the problem rather than a serious recommendation of how things should be done.
Post reply on HN