Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pipenv: Promises a Lot, Delivers Little (2020)

chriswarrick.com

Post reply on HN
Print view View original

61–68 of 68 posts

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#61

Post by plonk »

cuteboy19 wrote: post #52
plonk wrote: post #39

Earlier quoted context omitted.

> My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities. That happens very rarely with our dozens of transitive depe…

>That happens very rarely with our dozens of transitive dependencies Because you don't do ml. My requirements.txt is useless in 6 months because the transitive dependencies all have incompatible versions of common libraries by now

All we do is ML and image processing. We use huge libraries like PyTorch, ones with insane packaging like OpenCV, and things are so stable I haven’t felt the need to introduce version pinning yet.

Maybe you’re using recent less-stable projects? I found that the main ML packages rarely break things without months of deprecation warnings.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#62

Post by bamboozled »

radus wrote: post #59
bamboozled wrote: post #31

Earlier quoted context omitted.

PDM is based on PEP-582, which is only a draft and you will hit edge cases where it's not supported, or projects refuse to support it because of this. I'd avoid it for that reason personally. virtualenvs are a much more supported.

You can opt out of PEP-582, and in PDM 2.0 (just released), it becomes opt in.

What’s the point of using it over out Poetry then ? Why do we need a other ?

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#63

Post by bb88 »

wodenokoto wrote: post #10

Is there a reliable source that tells what the current recommended way of doing this in Python is? I’ve been using Anaconda and been finding that it install incompatible versions of jupyter and ipython dependencies and that a lot of tools I need only work from pip, so I’ve been wondering if maybe I should just switch to the “pythonic way of doing it” and have absolutely no idea what that is.

Today, we (and I mean me and my company) use poetry for prod code delivery and it just works. Poetry uses pip and virtualenv under the hood. It's worth understanding virtualenv regardless.

For every project you're developing, there will be virtualenv which has all the dependencies that project needs (which may be different than what's installed in the system, and different than what other projects may need).

"python -m venv init project/venv" will create it. "rm -rf project/venv" will delete it.

Usually the virtualenv goes somewhere well known, like "project/venv". Sourcing the activate script ( "source project/venv/bin/activate" ) changes your shell environment to use the virtualenv instead of the system python environment. Once activated "pip install package" installs to the activated virtualenv. "deactivate" will turn off the virtualenv.

This is semantic sugar for what's really happening behind the curtains. There's a copy of python in the virtualenv: "project/venv/bin/python" which runs in the virtual env regardless of whether the virtualenv is activated or not. "activate" just adds "project/venv/bin" to the start of the PATH. "deactivate" removes it.

Regardless you can always see which python you're using by typing "which python". System python (/usr/bin/python) will use system packages. The venv python (project/venv/bin/python) will use the virtualenv python packages.

This allows you to have different virtualenvs to try out things like new versions of python, say. And each virtualenv is isolated from every other virtualenv.

And poetry is just a nice wrapper around this process that also figures out total project dependencies and creates a "lock" file to freeze all deps to a particular version for consistent releases. Pipenv is basically the same thing.

"pip install --user poetry" will install it in your home directory.

It's not recommended to use conda and pip together, basically the recommendation is to use one or the other, though I've heard miniconda is better in this regard. YMMV.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#64

Post by solarkraft »

camgunz wrote: post #44
DanielVZ wrote: post #24

Earlier quoted context omitted.

First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…

> And finally the user interface is much more modern. I hear you on the pinning, but "UX" concerns like this drive me batty; I just use shell functions for this: venv() { . ${VIRTUALENV_FOLDER}/${1}/bin/activate } gitsu() { branch=$(git status | head -n 1 | cut -d ' ' -f 3) git push --set-upstream origin ${branch} } git_add_conflicts() { git add $(git status | grep 'both modified:' | cut -d ':' -f2) } It's so much ea…

But I would prefer something that just works without me having to do that. Chances are it'll also save me work in areas I'm not aware of yet.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#65

Post by ok_dad »

ptx wrote: post #57
partido3619463 wrote: post #55

Earlier quoted context omitted.

Constraints are not the same thing. Poetry locks the entire dependency closure. You can install with “no-deps” option, but then you have to specify specific versions of every dependency in requirements.txt

What do you mean? If you do "pip freeze >constraints.txt", that locks the versions of all installed packages, no matter where they came from. As an example, let's create a venv and install some older versions of Django and its dependencies (current versions are 0.4.2, 3.5.2 and 4.0.6) $ python3 -m venv env1 $ ./env1/bin/pip install sqlparse==0.4.0 asgiref==3.5.0 django=4.0.0 ...and also Flask just to complicate the c…

That’sa lot more than just

    poetry init
    poetry add x y z
Then later

    poetry install
And maybe

    poetry remove y

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#66

Post by smetj »

drekipus wrote: post #5

I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…

Totally agree on this.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#67

Post by camgunz »

solarkraft wrote: post #64
camgunz wrote: post #44

Earlier quoted context omitted.

> And finally the user interface is much more modern. I hear you on the pinning, but "UX" concerns like this drive me batty; I just use shell functions for this: venv() { . ${VIRTUALENV_FOLDER}/${1}/bin/activate } gitsu() { branch=$(git status | head -n 1 | cut -d ' ' -f 3) git push --set-upstream origin ${branch} } git_add_conflicts() { git add $(git status | grep 'both modified:' | cut -d ':' -f2) } It's so much ea…

But I would prefer something that just works without me having to do that. Chances are it'll also save me work in areas I'm not aware of yet.

Oh me too, I love when a tool solves problems I didn't even know I had, until I have them, I look for a solution, and poof Vim (or whatever) has already solved it in a coherent way.

If every tool were like that that would be one kind of world, but we live in one that's more interesting. There's some Mark Twain quite like "the world owes you nothing; it was here first", or maybe the Buddha saying "it's harder to soften the earth than it is to wear sandals". I like to think that sandal-making is a core engineering skill, whether you're a carpenter or a computer engineer.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#68

Post by strbean »

aden1ne wrote: post #12
strbean wrote: post #6

Earlier quoted context omitted.

How is the maturity/stability of Poetry these days? I despise Pipenv, and was hoping to push for a switch to Poetry at my place of work a couple years ago, but I ran into blocking bugs across multiple versions (latest N versions affected by bug A, prior M versions affected by bug B). Had to chock it up to "not yet mature enough" and resign myself to the absurd lock times and countless terrible behaviors of Pipenv.

We use poetry for all python projects. I haven't seen an actual poetry internal bug in quite a while, but using poetry effectively does require one keep some things in mind that are probably non-obvious to newcomers: 1. Poetry's default assumption on packages respecting semver simply does not hold up in reality. There are very few packages actually sticking to semver. Thus the `^x.y.z` default version range is quite…

I'll take the behavior of 2 over Pipenv's "any re-lock aggressively updates everything" approach. What's the point of lock files if you have to peg everything in order to have stability / control over versions??
Post reply on HN