Obligatory XKCD: https://xkcd.com/1987/
One does not simply 'pip install'
71–80 of 118 posts
Re: One does not simply 'pip install'
#72Earlier quoted context omitted.
Nowhere in the official Python documentation (where 99% of new python users are going to go) does it warn or even talk about Linux and Debian specific issues like only using apt packaged versions of dependencies. It wasn't even until recent years that pip gave a hint or warning something might break in those setups. The situation with Python on Debian has been pretty bad IMHO with a cloistered group of people saying…
"The situation with Python on Debian has been pretty bad IMHO" The situation with anything has been pretty bad in Debian lately. I'm all for the minimalistic approach in regards to Python. It's Ok to provide only the packages needed by applications and the core system. For everything else, there's pip. EDIT: I've meant to say, there's pip inside a venv.
I agree, the problem is if you just 'pip install foo' on Debian, like 99% of python docs and readmes say, it instantly fails. Worse yet if you think you should just elevate to root and force pip install to work you can potentially break the Python install. It's a nasty footgun.
Re: One does not simply 'pip install'
#73Earlier quoted context omitted.
The official python packaging documentation basically says, "here are 12 different tools that can do it, figure it out!" https://packaging.python.org/en/latest/tutorials/managing-de...
>> While pip alone is often sufficient for personal use, Pipenv is recommended for collaborative projects as it’s a higher-level tool that simplifies dependency management for common use cases. There are many, but it recommends one. I don't think any reasonable person will actually go out and try all 7(?) of them.
Which is basically the same advice. If you don’t already know what you want start with pip and requirements.txt and when you hit pain points: wanting to soft lock transitive dependencies, automate venv setup for onboarding, locking packages by hash, package caching, bundling Python itself with your code, etc. then there’s probably a tool out there tailored to your use-case.
Re: One does not simply 'pip install'
#74This post points out one of my struggles with python. I am not a python developer but I use python heavily for some tooling. So all I need to do is to “distribute” my tools to other servers in a replicable and consistent matter, isolated from global packages. Can you please help me understand two points? 1. If I use venv+pip to install some python app, do I have to “activate” that specific virtual environment before…
1: usually you can just run the binary by its path. tbh I don't fully understand why it doesn't always work, but it's fairly rare, and most of the ones I can kinda-remember may have been during install time.
2: due to 1, symlinks often work. It's how I've installed all of my custom python binaries. Otherwise you'll very frequently see python "binaries" installed by e.g. Homebrew that are actually ~5 lines of minor environment prep and then running the actual binary - that's the only reliable way afaik.
Bonus answer to 2: pipx looks pretty decent.
Re: One does not simply 'pip install'
#75The next Debian/Ubuntu releases will no longer allow `pip install` outside of a venv: https://discuss.python.org/t/pep-668-marking-python-base-env... You can still force it via `pip install --break-system-packages ...` if needed.
Good. Now we just need to get pip itself updated so it refuses to run outside of a venv, and refuses to run unless invoked with "python -m pip" and we'll finally have something at least half decent.
And don't even get me started about how much better npm is at publishing packages, versus pip's refusal to add the same user friendliness.
Re: One does not simply 'pip install'
#76This post points out one of my struggles with python. I am not a python developer but I use python heavily for some tooling. So all I need to do is to “distribute” my tools to other servers in a replicable and consistent matter, isolated from global packages. Can you please help me understand two points? 1. If I use venv+pip to install some python app, do I have to “activate” that specific virtual environment before…
1. You can call it directly by referencing the venv python exe eg /pqth/to/your/venv/bin/python /path/to/your/script.py
Re: One does not simply 'pip install'
#77Earlier quoted context omitted.
python3 includes "venv". If you don't have an existing preference, use that. python3 -m venv ../my-venv-dir # wherever you like . ../my-venv-dir/bin/activate pip install whatever you can close your terminal and "rm -r" the venv dir, and no trace will be left. (or you can just "deactivate" and use it again later)
The author mentioned venv in the article. Also I believe the parent comment is talking about the difficulties of choosing between different dependencies management solutions, venv among them, rather than the lack of (a good) one
Why cause yourself difficulty by drifting towards optionality vs. using the ops suggestion and using venv?
This topic gets posted to HN far too often - I'm starting to think people are deliberately avoiding venv for some reason, because otherwise it's a perfectly capable system for package management.
Re: One does not simply 'pip install'
#78Earlier quoted context omitted.
> And then node_modules, which packages should not rely on but do, Isn't the point of node_modules to house ... dependencies? I'm confused as to what you're getting at here.
I think he prefers a python-esque way where they're sort of dumped in a flat namespace (and not in current project directory), rather than the node_modules way where it's recursively a copy of each thing and its specific exact dependencies, all the way down. There are ways to not use node_modules, by using newer Yarns for example.
> There are ways to not use node_modules, by using newer Yarns for example.
My point was that if you use yarn2 in pmp mode, and you have a dependencies that depends on the node_modules layout being at the same level as package.json, than even if your package manager doesn't not need or use node_modules, it must emulate it so the dependencies can find their files.
Re: One does not simply 'pip install'
#79The article talks about installing Python packages for development, but if you find yourself using `pip` to install Python tools/scripts then you should use `pipx` - it will properly sandbox those tools so they don't break (or be broken by) the system or other Pythons: https://pypa.github.io/pipx/
The problem is that everyone has different problems with python packaging and everyone has different idea what you "should" do.
The core problem (as I see it) is that Linux distros tend not to have any firm distinction between "system" packages, "user" packages, and "development" packages (which are a subset of user packages). The system package manager installs everything globally, while also being considered the only approved/safe way to install packages.
Languages tend to try to get around this by providing their own package registries and build systems to use them (npm, pip, cargo, etc), and developer tools often include some sort of sandboxing to avoid interference from the system packages (venv, bazel, cargo, nix develop, etc).
For user packages a tool like Snap, home-manger, Flatpak, or AppImage seems necessary.
Python makes the problems very obvious, especially since it has so many package management systems, gets used for system packages, and gets used for user applications.
Re: One does not simply 'pip install'
#80> There’s no shortage of package management alternatives available for Python [...] > How someone is meant to pick between these as a new developer is a mystery. This. Every time I get booked to look at some Python project hours are usually wasted initially figuring out what dependency mgmt solution was used how. And with what 'special sauce' the resp. developers deemed to be 'the right way' (or some library required…
People are making it seem harder than necessary.
Just create a virtual environment and install your packages there.
Done.