Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

One does not simply 'pip install'

ianwootten.co.uk

Post reply on HN
Print view View original

51–60 of 118 posts

Re: One does not simply 'pip install'

#51

Post by qbasic_forever »

wheelerof4te wrote: post #17
lazka wrote: post #11

The next Debian/Ubuntu releases will no longer allow `pip install` outside of a venv: https://discuss.python.org/t/pep-668-marking-python-base-env... You can still force it via `pip install --break-system-packages ...` if needed.

The way it's meant to be. On Linux, you either use the system packages via "apt install", or you use venvs. EDIT: For context, I've meant "managed" distros like Debian and Ubuntu.

Nowhere in the official Python documentation (where 99% of new python users are going to go) does it warn or even talk about Linux and Debian specific issues like only using apt packaged versions of dependencies. It wasn't even until recent years that pip gave a hint or warning something might break in those setups. The situation with Python on Debian has been pretty bad IMHO with a cloistered group of people saying the status quo is just fine because it works for them exclusively.

Re: One does not simply 'pip install'

#52

Post by zokier »

hungryforcodes wrote: post #22

npm certainly has a number of problems (at the end the article compares pip to npm) -- but after reading this article I didn't realize pip was so problematic. I also didn't realize it installed things globally. So the solution is?

> So the solution is?

Always use venv.

Re: One does not simply 'pip install'

#53

Post by ziml77 »

kgwgk wrote: post #33

> You might expect if I were to pip uninstall requests that I get back to a clean system, right? Why would i expect that? If one day I install A and another day I install B, which depends on A, I wouldn’t expect to lose A of I were to uninstall B.

If I didn't have A installed and then I install B which transitively installs A, then I expect that uninstalling B will also uninstall A. If only one system is managing the packages, then it is able to do this. It will have a record of the things I've explicitly installed so it knows what dependencies are safe to uninstall.

Re: One does not simply 'pip install'

#54

Post by hnfong »

qbasic_forever wrote: post #45
Spivak wrote: post #37

Earlier quoted context omitted.

To me there’s no mystery. Just use pip and then when you hit a specific problem see if any existing tools solve it. The game of, “blank project immediately needs 12 different tools I read about on a blog post” is silly.

The official python packaging documentation basically says, "here are 12 different tools that can do it, figure it out!" https://packaging.python.org/en/latest/tutorials/managing-de...

>> While pip alone is often sufficient for personal use, Pipenv is recommended for collaborative projects as it’s a higher-level tool that simplifies dependency management for common use cases.

There are many, but it recommends one. I don't think any reasonable person will actually go out and try all 7(?) of them.

Re: One does not simply 'pip install'

#55

Post by KronisLV »

unxdfa wrote: post #21
lazka wrote: post #11

The next Debian/Ubuntu releases will no longer allow `pip install` outside of a venv: https://discuss.python.org/t/pep-668-marking-python-base-env... You can still force it via `pip install --break-system-packages ...` if needed.

This makes me so happy. Back when we had Jenkins slaves, one of our devops guys set a pipeline up that pip installed different versions over the top of system packages causing weird intermittent failures everywhere. Different pipelines would be running in different requirements files. I revoked sudo privs immediately for Jenkins (I didn't add them in the first place) and reprovisioned the whole build cluster resultin…

> Back when we had Jenkins slaves, one of our devops guys set a pipeline up that pip installed different versions over the top of system packages causing weird intermittent failures everywhere.

Not everyone might like containers, but using them for CI seems like a good way to avoid situations like this, at least when viable (e.g. web development). You get to choose what container images you need for your build, do whatever is necessary inside of them and they're essentially thrown away after once you're done with what you need to do, cache aside. They also don't have any significant impact or dependencies on the system configuration either, as long as you have some sort of a supported container runtime.

Re: One does not simply 'pip install'

#56

Post by qbasic_forever »

forgotpwd16 wrote: post #40
meitham wrote: post #28

Earlier quoted context omitted.

Hopefully that’s not going to be the case inside a container!

Why? Can overwrite it and even if couldn't making a new venv is just a `python -m venv venv` away.

Venv in a container is unnecessary ritual. It's a container, it has its own entire root filesystem...

Re: One does not simply 'pip install'

#57

Post by ZGDUwpqEWpUZ »

kgwgk wrote: post #33

> You might expect if I were to pip uninstall requests that I get back to a clean system, right? Why would i expect that? If one day I install A and another day I install B, which depends on A, I wouldn’t expect to lose A of I were to uninstall B.

apt handles this by marking packages as manually installed. You and the author could both be happy with that solution but afaik pip doesn't currently store such information.

Re: One does not simply 'pip install'

#58

Post by tyingq »

It's also interesting how things like AWS Lambdas, Graviton, etc, are exposing all the shortcomings of the various pip install, venv, poetry, etc, approaches.

It's not impossible to figure it out, but you end up spending a lot of time to come up with something that works locally, within containers, inside a CI/CD system, and then deployed out across things like Lambdas, or non x64 machines.

Then, after it's all working, upgrading the Python version, or an extension that has C code, etc, repeats some of the hard bits.

Re: One does not simply 'pip install'

#59

Post by hnfong »

unxdfa wrote: post #21
lazka wrote: post #11

The next Debian/Ubuntu releases will no longer allow `pip install` outside of a venv: https://discuss.python.org/t/pep-668-marking-python-base-env... You can still force it via `pip install --break-system-packages ...` if needed.

This makes me so happy. Back when we had Jenkins slaves, one of our devops guys set a pipeline up that pip installed different versions over the top of system packages causing weird intermittent failures everywhere. Different pipelines would be running in different requirements files. I revoked sudo privs immediately for Jenkins (I didn't add them in the first place) and reprovisioned the whole build cluster resultin…

> I revoked sudo privs immediately for Jenkins (I didn't add them in the first place)

If you allowed sudo in your jenkins jobs you're morally barred from blaming python for screwing up the system.

Re: One does not simply 'pip install'

#60

Post by mshockwave »

ploxiln wrote: post #50
virtualritz wrote: post #27

> There’s no shortage of package management alternatives available for Python [...] > How someone is meant to pick between these as a new developer is a mystery. This. Every time I get booked to look at some Python project hours are usually wasted initially figuring out what dependency mgmt solution was used how. And with what 'special sauce' the resp. developers deemed to be 'the right way' (or some library required…

python3 includes "venv". If you don't have an existing preference, use that. python3 -m venv ../my-venv-dir # wherever you like . ../my-venv-dir/bin/activate pip install whatever you can close your terminal and "rm -r" the venv dir, and no trace will be left. (or you can just "deactivate" and use it again later)

The author mentioned venv in the article. Also I believe the parent comment is talking about the difficulties of choosing between different dependencies management solutions, venv among them, rather than the lack of (a good) one
Post reply on HN