Pipenv: Promises a Lot, Delivers Little (2020) • Hacker News Board

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#51

Post by camgunz » Sat, Jul 16, 2022, 4:49 PM UTC

traverseda wrote: post #17

throwaway234232 wrote: post #15

Earlier quoted context omitted.

You don't need virtual environments when using npm or composer or cargo. They also create lock files with hashes by default. pip still doesn't use lock files. It's subpar compared to other ecosystems.

I worry that the inclusion of lockfiles will make semver less relevant. You shouldn't need to use lock files if you're using semver properly. If your ecosystem is healthy than pinning exact versions with lock files shouldn't be done. Making it so that every dev uses the latest patch or minor when they run your program. Using lock files for libraries should absolutely never happen, you should at most fix your dependen…

I've experienced multiple minor and patch (according to semver) updates that broke APIs and behavior, and I'd guess most devs have as well.

I think semver makes sense to humans. I can derive a lot of meaning from it when I see n.n.n. But when it comes to the software supply chain, it's just too rickety. Frankly, when you lose customers after a new deploy broke one of your dependencies, "but the dependency author didn't respect semver" isn't an excuse.

I say this as someone who strongly pushed `dependency>=1.3.2,<1.4` until that happened to me. My argument was "security updates", and now I just don't care. The software supply chain is too chaotic, and you have to be defensive against it.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#52

Post by cuteboy19 » Sat, Jul 16, 2022, 5:33 PM UTC

plonk wrote: post #39

mirashii wrote: post #23

Earlier quoted context omitted.

I really couldn’t disagree with this more. Reproducibility is far more important than integrating unknown bug fixes, and upgrading dependencies should be a manual process during which you review the changes and verify that they don’t break your system. My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versio…

> My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities. That happens very rarely with our dozens of transitive depe…

>That happens very rarely with our dozens of transitive dependencies

Because you don't do ml. My requirements.txt is useless in 6 months because the transitive dependencies all have incompatible versions of common libraries by now

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#53

Post by hibbelig » Sat, Jul 16, 2022, 5:40 PM UTC

throwaway234232 wrote: post #15

drekipus wrote: post #5

I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…

You don't need virtual environments when using npm or composer or cargo. They also create lock files with hashes by default. pip still doesn't use lock files. It's subpar compared to other ecosystems.

The difference between virtual environments in Python and node_modules in JavaScript is minor I would say. It’s the same concept. I believe composer has something similar. It’s a directory where packages go and they are not part of the system wide installation.

I agree that lock files are useful and it’s a pity that pip does not offer them.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#54

Post by bit-rot » Sat, Jul 16, 2022, 5:51 PM UTC

What is the currently accepted way of nailing down a specific python version if not using something like pipenv?

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#55

Post by partido3619463 » Sat, Jul 16, 2022, 6:28 PM UTC

ptx wrote: post #48

DanielVZ wrote: post #24

Earlier quoted context omitted.

First of all some years ago pip wasn't really solving dependencies. Then even with the new dependency resolving improvements it's way worse than poetry's in my experience. Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned…

> Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned dependencies will be compiled into a poetry.lock file. This has now been solved [1] in pip, so no third-party tool is needed. Put your pinned dependencies in constraints…

Constraints are not the same thing. Poetry locks the entire dependency closure. You can install with “no-deps” option, but then you have to specify specific versions of every dependency in requirements.txt

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#56

Post by alanwreath » Sat, Jul 16, 2022, 6:38 PM UTC

Seems like instead of focusing on python's execution speed, we've all been saying (for a long time) what we really really need in a language (like python) is simplified/centralized/pythonic(one way to do it) tooling. Python, isn't popular because it executes close to C speeds. It's popular because it generally removes common obstacles for the developer. The package manager for python is a common obstacle that many open source repo's are trying to solve because python proper hasn't solved it (or attempted to with enough focus).

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#57

Post by ptx » Sat, Jul 16, 2022, 6:43 PM UTC

partido3619463 wrote: post #55

ptx wrote: post #48

Earlier quoted context omitted.

> Also the requirements.txt file tends to get cluttered with non top level dependencies that make upgrading your dependencies an herculean task. Instead on poetry you define your top level dependencies in your pyproject.toml and all the actual pinned dependencies will be compiled into a poetry.lock file. This has now been solved [1] in pip, so no third-party tool is needed. Put your pinned dependencies in constraints…

Constraints are not the same thing. Poetry locks the entire dependency closure. You can install with “no-deps” option, but then you have to specify specific versions of every dependency in requirements.txt

What do you mean? If you do "pip freeze >constraints.txt", that locks the versions of all installed packages, no matter where they came from.

As an example, let's create a venv and install some older versions of Django and its dependencies (current versions are 0.4.2, 3.5.2 and 4.0.6)

  $ python3 -m venv env1
  $ ./env1/bin/pip install sqlparse==0.4.0 asgiref==3.5.0 django=4.0.0

...and also Flask just to complicate the constraints file for the example:

  $ ./env1/bin/pip install flask

Lock all dependency versions in constraints.txt:

  $ ./env1/bin/pip freeze >constraints.txt

Create a requirements file that specifies just "django" and references the constraints file:

  $ echo '-c constraints.txt' >requirements.txt
  $ echo 'django' >>requirements.txt

  $ cat requirements.txt 
  -c constraints.txt
  django

  $ cat constraints.txt 
  asgiref==3.5.0
  click==8.1.3
  Django==4.0
  Flask==2.1.3
  itsdangerous==2.1.2
  Jinja2==3.1.2
  MarkupSafe==2.1.1
  sqlparse==0.4.0
  Werkzeug==2.1.2

Now we can create another venv with the exact same versions of Django and all its dependencies (but not Flask or its dependencies) using just pip and the requirements file:

  $ python3 -m venv env2
  $ ./env2/bin/pip install -r requirements.txt 
  $ ./env2/bin/pip freeze
  asgiref==3.5.0
  Django==4.0
  sqlparse==0.4.0

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#58

Post by potatochup » Sat, Jul 16, 2022, 7:17 PM UTC

bvrmn wrote: post #14

jks wrote: post #8

Earlier quoted context omitted.

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

pip freeze > stable-reqs.txt ??

This kinda works, but it isn't a cross-platform solution.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#59

Post by radus » Sat, Jul 16, 2022, 8:28 PM UTC

bamboozled wrote: post #31

greut wrote: post #2

It's a bit of an old post, and yes Pipenv is not the go-to tool anymore. pip-tools is okay for people that really, really love their requirements.txt; otherwise we tend to go with Poetry at work. Any folks having a good experience with PDM https://github.com/pdm-project/pdm ?

PDM is based on PEP-582, which is only a draft and you will hit edge cases where it's not supported, or projects refuse to support it because of this. I'd avoid it for that reason personally. virtualenvs are a much more supported.

You can opt out of PEP-582, and in PDM 2.0 (just released), it becomes opt in.

Re: Pipenv: Promises a Lot, Delivers Little (2020)

#60

Post by Macha » Sat, Jul 16, 2022, 8:34 PM UTC

drekipus wrote: post #5

I'm really sorry, but I can never quite understand what the deal is with these tools are. I understand that virtual environments are important, but how are these tools any better than python3 -m venv .venv --prompt="foobar" . .venv/bin/activate pip install -r requirements.txt ? Whenever I see all these other tools I just get the feeling like there's some big elephant in the room that everyone is battling against, but…

What do you list in your requirements.txt? If it's just the top level dependencies, you could have versions change of your transitive dependencies which can matter and break things. If it's all dependencies, you lose the context of what you need because you use it and what you need just because it's a dependency of a dependency.

Some projects used to have two files for this, effectively managing a lockfile manually with pip freeze, but then it's nice to have a wrapper around this pattern and that's where the first gen like pip-tools came from, and then stuff like poetry/pipenv is aiming to streamline that (and avoid manual use of companions like pyenv for specific Python versons) even more.