Live data from Hacker News

Freezing Python’s Dependency Hell

tech.instacart.com

71–80 of 152 posts

Re: Freezing Python’s Dependency Hell

#71

What's wrong with pipenv? I am genuinely curious. On local : mkdir my_project_directory cd my_project_directory export PIPENV_VENV_IN_PROJECT=1 (To make the virtual environment folder determininstic(.venv/) otherwise you will get a hash based directory(my_project_directory-some-hash-value) which might not be suitable for automatic deployments in applications like docker. I don't know why this is not default.) pipenv…

Also, for now, pipenv handles badly custom pipy with authentication

Re: Freezing Python’s Dependency Hell

#72
ruby practices based around bundler aren't perfect, but they did solve _this_ level of problem ~7 years ago.

It remains a mystery to me why python seems to have won the popularity battle against ruby. They are very similar languages, but in all ways they differ ruby seems superior to me.

Re: Freezing Python’s Dependency Hell

#73

genuine question - is nobody using anaconda/conda in production ? I have found the binary install experience in conda far more pleasant than in anything else. Going forward, the trend is going to be pipenv+manylinux ( https://github.com/pypa/manylinux ), but conda is super pleasant today

Why are you moving away from conda going forward?

we are NOT moving away. In fact, for all its faults, conda has been the most pleasant experience. We publish our own internal packages as conda packages.

I was commenting about manylinux becoming an official PEP - https://www.python.org/dev/peps/pep-0513/ - could eventually end up supplanting conda.

I wish they had adopted conda itself. Because manylinux was clearly inspired by conda.

"Instead, we define a standard subset of the kernel+core userspace ABI that, in practice, is compatible enough that packages conforming to this standard will work on many linux systems, including essentially all of the desktop and server distributions in common use. We know this because there are companies who have been distributing such widely-portable pre-compiled Python extension modules for Linux -- e.g. Enthought with Canopy [4] and Continuum Analytics with Anaconda [5].

Building on the compatibility lessons learned from these companies, we thus define a baseline manylinux1 platform tag for use by binary Python wheels, and introduce the implementation of preliminary tools to aid in the construction of these manylinux1 wheels."

Re: Freezing Python’s Dependency Hell

#74

Anybody played with the brand new XAR from Facebook? https://code.fb.com/data-infrastructure/xars-a-more-efficien...

Thanks for the link. That looks interesting; I'll have to give that a try. When I started reading the link my first thought was Pex from Twitter. I don't know how comparable XAR is to Pex but it's worth a look to compare the two.

Re: Freezing Python’s Dependency Hell

#75

Earlier quoted context omitted.

Is there a reason you don't "activate" your virtualenv? That (with the addition of using mkvirtualenv and friends) is the workflow I use to both dev and prod and am really happy with!

I hate the whole idea of activating virtualenvs. It's a tool that makes it really easy to end up running a command in the wrong environment and see weird behavior instead of a clear error message. I've seen variations on this scenario happen at least 3 times, for instance: 1) Somebody creates script that activates and runs django and commits it. 2) Junior runs script but the virtualenv doesn't get created for some re…

> 1) Somebody creates script that activates and runs django and commits it.

You can call the python bin inside the virtualenv and it will run as if the virtualenv was active:

  venv/bin/python -m foo.bar
Obviously it doesn't work if devs used different names for their virtualenvs. Work has a convention to always use the same name so this works pretty well.

Re: Freezing Python’s Dependency Hell

#76
post #66

What's wrong with pipenv? I am genuinely curious. On local : mkdir my_project_directory cd my_project_directory export PIPENV_VENV_IN_PROJECT=1 (To make the virtual environment folder determininstic(.venv/) otherwise you will get a hash based directory(my_project_directory-some-hash-value) which might not be suitable for automatic deployments in applications like docker. I don't know why this is not default.) pipenv…

Last time I tried, it also required that the target Python version be installed somewhere on the path. If pipenv used venv instead of virtualenv, something like pyenv to retrieve/install Python versions, and was distributed as a full executable (rather than requiring a bootstrapped Python) I would actually it.

`pipenv install --python ~/.pyenv/versions/3.6.5/bin/python` works for me, and that directory is not on my path. Did you try the more explicit `--python` flag?

Re: Freezing Python’s Dependency Hell

#77
post #38

Earlier quoted context omitted.

I was of the same mentality, until yesterday when I watched this PyCon video, uploaded 13 May 2018. https://www.youtube.com/watch?v=GBQAKldqgZs To cut a long story short, if you're happy with virtualenv and pip then that's great, but the idea of pipenv is to replace virtualenv and pip, which means you'll actually have one tool fewer. :)

Genuine question: does pipenv do anything that [mini]conda doesn't?

Does conda create a lockfile?

Re: Freezing Python’s Dependency Hell

#78
Dependency hell in Python ? The only annoying part would be missing some library to build certain packages, like lxml, etc.

That's all.

We Python developers are fortunate to have amazing tools such as pip, virtualenv, etc.

Re: Freezing Python’s Dependency Hell

#79
post #13

1. Build Docker image out of requirements.txt 2. Develop application 3. Repeat 1-2 until ready to deploy 4. Run Docker image in production with same dependencies as development 5. ?? 6. Profit! As long as you don't rebuild in between steps 3-4, you'll have the same set of dependencies down to the exact patch level.

Doesn't help developers not get different versions of packages. Lockfiles are necessary regardless of Docker.

Re: Freezing Python’s Dependency Hell

#80
post #66

What's wrong with pipenv? I am genuinely curious. On local : mkdir my_project_directory cd my_project_directory export PIPENV_VENV_IN_PROJECT=1 (To make the virtual environment folder determininstic(.venv/) otherwise you will get a hash based directory(my_project_directory-some-hash-value) which might not be suitable for automatic deployments in applications like docker. I don't know why this is not default.) pipenv…

Last time I tried, it also required that the target Python version be installed somewhere on the path. If pipenv used venv instead of virtualenv, something like pyenv to retrieve/install Python versions, and was distributed as a full executable (rather than requiring a bootstrapped Python) I would actually it.

pipenv will do automatic python installation if you have pyenv installed [1]. Pyenv isn't bundled in the base install, but I've been using them both and have been happy switching between environments with different Python versions.

[1]: https://docs.pipenv.org/advanced/#automatic-python-installat...

Post reply on HN