[deleted]
Pipenv: Promises a Lot, Delivers Little (2020)
41–50 of 68 posts
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#42Earlier quoted context omitted.
First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…
Are you guys actually using either poetry or venv on production? Separately, I'm curious to hear examples of where dependency-clutter actually caused a problem.
We use poetry just for development, but run Docker containers in prod. When the image gets built we just create a requirements.txt (poetry export --format requirements.txt --output requirements.txt), copy that into the image, and pip install. Because this is built using the poetry lock file, it'll always be exactly the same unless we specifically update something with poetry.
I used to work at a place that was just using requirements.txt files that only included our direct dependencies. There was a project that needed updating after not being touched for a couple of years. The requirements.txt didn't change, but when we built the project again, some of the transitive dependencies used a newer version, and a bug was introduced from one of those updates. A bunch of time was wasted tracking down the issue, pinning the old version of the transitive dependency, and figuring out the damage caused by the bug.
As a result, the requirements.txt was changed to also include transitive dependencies. We had vulnerability scanning on our code, and it found a severe issue with one of the transitive dependencies, but there wasn't a version of that library with the issue fixed yet. Time was spent looking into this to see how we could be impacted. As it turns out, it was a transitive dependency for a library that we no longer used and removed from the project months ago. When you create your requirements.txt by running pip freeze > requirements.txt, you don't have an easy way of knowing which library requires which transitive dependency.
There's ways you can fix this using multiple requirements.txt files, but at that point it's a lot easier to use poetry, especially if you want to keep your development dependencies separate.
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#43Earlier quoted context omitted.
How is the maturity/stability of Poetry these days? I despise Pipenv, and was hoping to push for a switch to Poetry at my place of work a couple years ago, but I ran into blocking bugs across multiple versions (latest N versions affected by bug A, prior M versions affected by bug B). Had to chock it up to "not yet mature enough" and resign myself to the absurd lock times and countless terrible behaviors of Pipenv.
We use poetry for all python projects. I haven't seen an actual poetry internal bug in quite a while, but using poetry effectively does require one keep some things in mind that are probably non-obvious to newcomers: 1. Poetry's default assumption on packages respecting semver simply does not hold up in reality. There are very few packages actually sticking to semver. Thus the `^x.y.z` default version range is quite…
Updating dependencies (#2) does seem needlessly painful. I have wondered if I am missing some obvious workflow.
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#44I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…
First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…
> And finally the user interface is much more modern.
I hear you on the pinning, but "UX" concerns like this drive me batty; I just use shell functions for this:
venv() {
. ${VIRTUALENV_FOLDER}/${1}/bin/activate
}
gitsu() {
branch=$(git status | head -n 1 | cut -d ' ' -f 3)
git push --set-upstream origin ${branch}
}
git_add_conflicts() {
git add $(git status | grep 'both modified:' | cut -d ':' -f2)
}
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#45Earlier quoted context omitted.
pip freeze > stable-reqs.txt ??
https://github.com/pypa/pip/issues/4732
This thread is just about pip freeze not including hashes, and while it's true that would provide a lot of reassurance, you can still pin transitive version dependencies w/o them.
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#46I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…
The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.
This is solved in pip nowadays with constraint files [1], so you can put your top-level dependencies in requirements.txt and all the transient ones in constraints.txt:
pip install -r requirements.txt -c constraints.txt
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#47Earlier quoted context omitted.
I really couldn’t disagree with this more. Reproducibility is far more important than integrating unknown bug fixes, and upgrading dependencies should be a manual process during which you review the changes and verify that they don’t break your system. My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versio…
> My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities. That happens very rarely with our dozens of transitive depe…
Nothing causes trouble often, because the class of problems is eliminated by pinning.
Still, it doesn’t have to happen often to be a serious hindrance to either developer productivity or your business. Imagine trying to roll out a fix for a production outage only to find a random transitive dependency has started breaking your build. Now your outage is extended to the duration of debugging and fixing an unrelated and avoidable problem.
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#48I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…
First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…
> Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned dependencies will be compiled into a poetry.lock file.
This has now been solved [1] in pip, so no third-party tool is needed. Put your pinned dependencies in constraints.txt and your top-level dependencies plus the line "-c constraints.txt" in requirements.txt.
[1] https://pip.pypa.io/en/stable/user_guide/#constraints-files
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#49I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…
[deleted]
Re: Pipenv: Promises a Lot, Delivers Little (2020)
#50Lots of folks are saying pipenv is outdated (and I agree), but it is still listed on the packaging docs [0] with little warning, and as the second tool on the official list of recommended tools [1]. If the community wants to deprecate this extremely undermaintained tool, we need to stop pointing people at it.
0: https://packaging.python.org/en/latest/key_projects/
1: https://packaging.python.org/en/latest/guides/tool-recommend...