Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pipenv: Promises a Lot, Delivers Little (2020)

chriswarrick.com

Post reply on HN
Print view View original

21–30 of 68 posts

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#21

Post by Hackbraten »

traverseda wrote: post #19
jks wrote: post #8

Earlier quoted context omitted.

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is n…

> That's a feature, you should be using the latest minor/patch release when developing.

You’re not wrong, but developers still need precise control about which version is pinned so they have a chance to figure out whether a bug is in their code or a regression in a dependency.

> If a dependency is constantly breaking on minor/patch releases you shouldn't be using it.

A more common example would be a dependency of decent quality, but there will still be bugs because there’s no such thing as bug-free software. And when the inevitable issue comes up, it’s super useful to have a switch that lets you pin down the regression and helps you figure out whether it’s on you or on the upstream project to fix the issue.

> Please don't lock your dependencies to specific minor/patch versions

Even if you allow a version range of "*" in your dependency file, it may still be a good practice for your app to have the resolved version number pinned in a dependency lock file, and even commit that lock file to version control.

> Especially when you're writing a library. If you're locking to anything other than a major release or a minimum minor/patch revision when producing a library things have gone very wrong.

You’re referring to the dependency file, not the dependency lock file, right? I couldn’t agree more. At the same time, it may still make sense for some libraries to have a lock file during the development process, and even commit that lock file to version control. You just can’t include that lock file in your release.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#22

Post by zarzavat »

traverseda wrote: post #19
jks wrote: post #8

Earlier quoted context omitted.

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is n…

I don't agree. Repeatability is more important. Every developer working on a project should have the same version of each dependency. The testing server should have the same version. Every deployment should have the same version.

I don't want my tools "helpfully" upgrading me to a different version, I don't care if it's "minor". It's different and that's a potential source of heisenbugs. Any change to dependencies should be an explicit action and it should be a commit in SCM.

By all means show an ugly warning message to nag people to upgrade but computers live to serve us, not the other way around. The moment you start devolving these decisions to a computer you've lost control of your core competency, which is ultimately what code is running on the machine.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#23

Post by mirashii »

traverseda wrote: post #19
jks wrote: post #8

Earlier quoted context omitted.

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is n…

I really couldn’t disagree with this more. Reproducibility is far more important than integrating unknown bug fixes, and upgrading dependencies should be a manual process during which you review the changes and verify that they don’t break your system.

My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities.

You can always upgrade overzealously when dependencies are pinned, falling back to the workflow you’ve described, but the inverse isn’t true. If your tools don’t allow you to pin, you’re signing yourself up for churn at unknown and unpredictable times.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#24

Post by DanielVZ »

drekipus wrote: post #5

I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…

First of all some years ago pip wasn't really solving dependencies.

Then even with the new dependency resolving improvements it's way worse than poetry's in my experience.

Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned dependencies will be compiled into a poetry.lock file.

And finally the user interface is much more modern. `poetry shell` is way simpler than `source venv/bin/activate`. You don't even need to run `poetry shell` because there's `poetry run`. And to install a new project is just `poetry install` instead of `python3 -m venv venv; source venv/bin/activate; pip install -r requirements.txt`.

Not to mention that it's way easier for me to understand where the depenency conflicts from console output compared to even the newest pip versions.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#25

Post by mirashii »

traverseda wrote: post #17
throwaway234232 wrote: post #15

Earlier quoted context omitted.

You don't need virtual environments when using npm or composer or cargo. They also create lock files with hashes by default. pip still doesn't use lock files. It's subpar compared to other ecosystems.

I worry that the inclusion of lockfiles will make semver less relevant. You shouldn't need to use lock files if you're using semver properly. If your ecosystem is healthy than pinning exact versions with lock files shouldn't be done. Making it so that every dev uses the latest patch or minor when they run your program. Using lock files for libraries should absolutely never happen, you should at most fix your dependen…

Semver promises are insufficient in a world where supply chain attacks are increasingly common. Pulling untested and invalidated code in at every project build is how that transitive dependency on a package that was taken over for a small window wrecks your development team. You should never be pulling in new code by surprise, it should always be something I’m aware of and signing up for.

The Rust ecosystem is good evidence that locking doesn’t kill semver. Semver is still widely used and has all of its meaning.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#26

Post by ramphastidae »

traverseda wrote: post #19
jks wrote: post #8

Earlier quoted context omitted.

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is n…

This simply doesn’t work on a large, shared codebase. Locking dependency versions is a must. You can’t just roll the dice that no breaking changes have been introduced by dependencies.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#27

Post by remram »

traverseda wrote: post #17
throwaway234232 wrote: post #15

Earlier quoted context omitted.

You don't need virtual environments when using npm or composer or cargo. They also create lock files with hashes by default. pip still doesn't use lock files. It's subpar compared to other ecosystems.

I worry that the inclusion of lockfiles will make semver less relevant. You shouldn't need to use lock files if you're using semver properly. If your ecosystem is healthy than pinning exact versions with lock files shouldn't be done. Making it so that every dev uses the latest patch or minor when they run your program. Using lock files for libraries should absolutely never happen, you should at most fix your dependen…

> You shouldn't need to use lock files if you're using semver properly.

You mean "if all your dependencies are using semver properly". Yes, you're right, and you've found the problem.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#28

Post by pen2l »

DanielVZ wrote: post #24
drekipus wrote: post #5

I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…

First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…

Are you guys actually using either poetry or venv on production?

Separately, I'm curious to hear examples of where dependency-clutter actually caused a problem.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#29

Post by donarb »

I've started to get away from virtual environments and just use Docker containers where all the pip dependencies can live as first class citizens. Also eliminates having to source the activate file or wonder if I'm calling the correct version of Python when I want to ssh in and run some code by hand in the container.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#30

Post by p4ul »

wodenokoto wrote: post #10

Is there a reliable source that tells what the current recommended way of doing this in Python is? I’ve been using Anaconda and been finding that it install incompatible versions of jupyter and ipython dependencies and that a lot of tools I need only work from pip, so I’ve been wondering if maybe I should just switch to the “pythonic way of doing it” and have absolutely no idea what that is.

I have seen the Hypermodern Python series of blog posts referenced here a few times. I'm a big fan of that approach personally.

I'd also be curious what others recommend!

https://cjolowicz.github.io/posts/hypermodern-python-01-setu...

Post reply on HN