Anybody played with the brand new XAR from Facebook?
https://code.fb.com/data-infrastructure/xars-a-more-efficien...
Freezing Python’s Dependency Hell
31–40 of 152 posts
Re: Freezing Python’s Dependency Hell
#32Doesn't using requirments.txt not account for (I forget the official name) Double Dependencies, you dependencies in requirements.txt might have a dependency whose version number may change over time.
This seems like something pip freeze could handle but doesn't.
Re: Freezing Python’s Dependency Hell
#33Never had a problem with dependencies in Python. Just keep it simple. When starting a new project: virtualenv venv -p *path-to-python-version-you-want* ./venv/bin/pip install *name-of-package* When running that project: ./venv/bin/python *name-of-python-file* Many people don't realize that the venv/bin/ contains all the relevant binaries with the right library path's out of the box.
Is there a reason you don't "activate" your virtualenv? That (with the addition of using mkvirtualenv and friends) is the workflow I use to both dev and prod and am really happy with!
I don't like "magic". I don't need anything to hijack PS1 and muck around with my shell.
Re: Freezing Python’s Dependency Hell
#34Earlier quoted context omitted.
Is there a reason you don't "activate" your virtualenv? That (with the addition of using mkvirtualenv and friends) is the workflow I use to both dev and prod and am really happy with!
I don't like "magic". I don't need anything to hijack PS1 and muck around with my shell.
Fair enough.
Re: Freezing Python’s Dependency Hell
#35What's wrong with pipenv? I am genuinely curious. On local : mkdir my_project_directory cd my_project_directory export PIPENV_VENV_IN_PROJECT=1 (To make the virtual environment folder determininstic(.venv/) otherwise you will get a hash based directory(my_project_directory-some-hash-value) which might not be suitable for automatic deployments in applications like docker. I don't know why this is not default.) pipenv…
A lot of people seem to run into bugs and some hit a brick wall when they report them: https://www.reddit.com/r/Python/comments/8elkqe/pipenv_a_gui... Personally I think poetry doesn't get enough visibility. It's not as hyped as pipenv but it feels a bit nicer: https://poetry.eustace.io/
It really serves a different audience/purpose.
Poetry replaces setup.py which is mostly used for building libraries. You still need to create your own virtualenv.
Pipenv replaces requirements.txt and handles the virtualenv for you. It can’t be used for packaging libs, but it’s primary purpose is to make developing on apps easier.
Lore seems to be much closer to pipenv than poetry.
Re: Freezing Python’s Dependency Hell
#36Why would you do this?
Redirect chain:
https://tech.instacart.com/freezing-pythons-dependency-hell-in-2018-f1076d625241
https://medium.com/m/global-identity?redirectUrl=https%3A%2F%2Ftech.instacart.com%2Ffreezing-pythons-dependency-hell-in-2018-f1076d625241
https://tech.instacart.com/freezing-pythons-dependency-hell-in-2018-f1076d625241?gi=85c0588ca374Re: Freezing Python’s Dependency Hell
#37genuine question - is nobody using anaconda/conda in production ? I have found the binary install experience in conda far more pleasant than in anything else. Going forward, the trend is going to be pipenv+manylinux ( https://github.com/pypa/manylinux ), but conda is super pleasant today
I use miniconda in production, and it's awesome. It's on par with (or even better than) npm except perhaps on the number of packages in the repository, supports pip, does everything I need and then some.
I'm baffled myself at the anaconda-blindness in the general crowd, which is evident every single time this comes up for discussion.
Re: Freezing Python’s Dependency Hell
#38Earlier quoted context omitted.
Thanks for the tip, but honestly, I don't need another tool.
I was of the same mentality, until yesterday when I watched this PyCon video, uploaded 13 May 2018. https://www.youtube.com/watch?v=GBQAKldqgZs To cut a long story short, if you're happy with virtualenv and pip then that's great, but the idea of pipenv is to replace virtualenv and pip, which means you'll actually have one tool fewer. :)
Genuine question: does pipenv do anything that [mini]conda doesn't?
Re: Freezing Python’s Dependency Hell
#39Never had a problem with dependencies in Python. Just keep it simple. When starting a new project: virtualenv venv -p *path-to-python-version-you-want* ./venv/bin/pip install *name-of-package* When running that project: ./venv/bin/python *name-of-python-file* Many people don't realize that the venv/bin/ contains all the relevant binaries with the right library path's out of the box.
Is there a reason you don't "activate" your virtualenv? That (with the addition of using mkvirtualenv and friends) is the workflow I use to both dev and prod and am really happy with!
I hate the whole idea of activating virtualenvs. It's a tool that makes it really easy to end up running a command in the wrong environment and see weird behavior instead of a clear error message.
I've seen variations on this scenario happen at least 3 times, for instance:
1) Somebody creates script that activates and runs django and commits it.
2) Junior runs script but the virtualenv doesn't get created for some reason.
3) The "warning virtualenv doesn't exist" message appears briefly and gets missed.
4) The junior gets "import error: cannot import django" or something.
5) They then start installing django in their system environment and... it sort of works. Except then they get another import error. And a whole bunch of python packages installed in their system environment. Yech.
Moreover, I'm really not sure what was so wrong with just running ./venv/bin/python in the first place and never having to worry about what environment you're in.
Re: Freezing Python’s Dependency Hell
#401. Build Docker image out of requirements.txt 2. Develop application 3. Repeat 1-2 until ready to deploy 4. Run Docker image in production with same dependencies as development 5. ?? 6. Profit! As long as you don't rebuild in between steps 3-4, you'll have the same set of dependencies down to the exact patch level.
This has the added benefit of letting you encode the system dependencies (OS packages) for library build time and for run time.
Docker images are also a great way to distribute Python CLI tools, certainly far better than installing via pip which either pollutes global state or is confined to a certain project's virtualenv.