Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use `Python -m Pip`

snarky.ca

Post reply on HN
Print view View original

41–50 of 141 posts

Re: Use `Python -m Pip`

#41

Post by shadycuz »

bxparks wrote: post #29

What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch easily among the various Python versions? Every time I try to research this, I get lost in the chaos of Python packaging and environments. Currently I do a `pip3 install -e .`, which uses the default Python provided by the OS. Then I hope that my continuous i…

You need `pyenv` and `nox`. Just read this entire guide https://cjolowicz.github.io/posts/hypermodern-python-01-setu...

Re: Use `Python -m Pip`

#42

Post by kbumsik »

AFAIR I needed to do that when upgrading pip itself on Windows.

`pip install --upgrade pip` may trigger error on Windows because the binary cannot be updated itself.

`python -m pip install --upgrade pip` works instead.

Re: Use `Python -m Pip`

#43

Post by butwhywhyoh »

I mean, this is generally good advice, but this reads like an infomercial where they show someone struggling REALLY hard to boil water to make a pot of spaghetti, when we all know it's really not that hard.

They try to make finding your pip executable sound difficult, and even more difficult would be to understand what interpreter its tied to. Except...

> pip -V

> # pip 19.0.3 from /usr/local/lib/python2.7/site-packages/pip (python 2.7)

I'm all for evangelizing best practices, but let's at least be honest about it.

Re: Use `Python -m Pip`

#44

Post by psyklic »

butwhywhyoh wrote: post #43

I mean, this is generally good advice, but this reads like an infomercial where they show someone struggling REALLY hard to boil water to make a pot of spaghetti, when we all know it's really not that hard. They try to make finding your pip executable sound difficult, and even more difficult would be to understand what interpreter its tied to. Except... > pip -V > # pip 19.0.3 from /usr/local/lib/python2.7/site-packa…

I think the point is that most people just immediately type `pip install ...`, without thinking about which interpreter it's tied to.

It's fairly common for beginners to tell me they installed something using pip, then the Python interpreter can't find the package.

Re: Use `Python -m Pip`

#45

Post by remram »

dragonwriter wrote: post #20
cmcconomy wrote: post #16

I wish there was a way to lock the global python so you couldn't install packages to it by accident

On unixy systems, won't the system Python usually require root/sudo to install packages, whereas the ones in your environments will be owned by a less-privileged user? And on Windows, you may have a globally installed Python (though there is little reason to have one instead of the Py launcher), but even if you do it's not a system Python that system components are relying on.

Unfortunately the pip developers had the genius idea to add a "user install" mode which pip will default to in this situation. So if you're not root, you won't be able to mess up your system-global Python, but you can still mess up your user-global Python (deleting some ~/.local subfolders takes care of this).

Re: Use `Python -m Pip`

#46

Post by bloblaw »

xdfgh1112 wrote: post #37
bloblaw wrote: post #18

Earlier quoted context omitted.

I remember when Perl was more popular generally than Python (late 90's and early 00's), and Perl espoused the mantra of (There is more than one way to do it: TIMTOWTDI). As a tongue-in-cheek reaction, Python espoused "TOOWTDI", or There's Only One Way To Do It :) reference: https://wiki.python.org/moin/TOOWTDI The problem is, when it comes to the Python package management ecosystem, there are SO MANY ways to do it. A…

I'm relatively new to python. I use venv, pip and requirements.txt. It's dead simple. What am I missing?

The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern...to me.

Python scripting today requires shipping your development environment. Python is wonderful until you want to run that code on another machine. At that point, the target system has to venv their way into reproducing your environment...often including the specific Python interpreter you picked, and to download (and possibly compile) all modules and their dependencies.

It can work beautifully and many of the large companies I've worked for have put oodles of engineering effort into making it "easy", as long as you follow their happy path and don't deviate.

But, to your point, on my own machine with venv + pip it "just works". The pain comes when trying to venv + pip on another machine.

How confident are you that you could venv + pip your moderately complex Python application on 200k machines without issue? What if it's a mix of Windows, Linux, and macOS? (this is a real scenario I've experienced). From my own experience I can share that it is painful. Your mileage may vary.

And after a while your system is riddled with Python venvs. Leading to some mildly annoying problems with popular IDE's: https://www.jetbrains.com/help/pycharm/package-installation-...

Re: Use `Python -m Pip`

#47

Post by bravetraveler »

kstrauser wrote: post #28
cmcconomy wrote: post #16

I wish there was a way to lock the global python so you couldn't install packages to it by accident

I have: $ cat ~/.pip/pip.conf [global] require-virtualenv = true ...which makes pip refuse to install anything unless I'm in an activated virtualenv. That, plus running as a regular user that doesn't have write permission to /usr, goes a long way.

I love this, pip is an absolute nightmare on systems with a restricted umask.

Newly created files/directories lack permissions for regular users... essentially breaking the interpreter for everyone [if /usr, sudo, etc]

Re: Use `Python -m Pip`

#48

Post by CaliforniaKarl »

It has only recently sunk in for me that, with the latest versions, Pip is not a tool for maintaining an environment and list of dependencies (where you'd have a lockfile, for example). Instead, it is an interface to…

1. Download packages from PyPi (or a different repository that provides the same interface)

2. Read the pyproject.toml file to find the build backend to use, and then install that.

3. Call the build backend to actually do the installation. This can be Poetry, Setuptools, Flit, or something else.

Pipenv is another such interface.

PEP 517 (https://peps.python.org/pep-0517/) created pyproject.toml and defined the API that build backends follow. That API includes a way for the build backend to tell Pip (or Pipenv, etc.) what dependencies to install. For systems that have a lockfile (like Pipenv), that could be a list of packages with explicit versions.

PIP 660 extended PEP 517, defining a standard way to have "editable installs". In other words, a way to support `pip install -e package`, or even `pip install -e .`.

The above is all my understanding, which is not at all authoritative!

It's worth checking out the list of packaging-related PEPs: https://peps.python.org/topic/packaging/ It's worth reading PEPs 518 and 621.

As an example, here's a Python package that uses Poetry: https://github.com/globus/globus-timer-cli It works perfectly fine with Pip, because Pip sees (via `pyproject.toml`) that it needs to pull in Poetry, and call it to do the actual install.

Here's an older repo of mine, where I've just started to learn about the transition: https://github.com/stanford-rc/mais-apis-python/blob/main/py... In this case, I could delete `pyproject.toml` entirely, Pip would see the `setup.py` file, and understand to use Setuptools. This was me when I was just starting to learn about the transition.

Finally, here's a newer repo of mine, where I've ditched setup.py (and _almost_ ditched setup.cfg) entirely: https://github.com/stanford-rc/globus-group-manager I'm still using Setuptools, but all of the metadata and requirements are included in the `pyproject.toml` file.

It's definitely been a rocky transition, but it's really looking (to me, at least) like we're at (or near) the point where I can just use `pip install …` and it'll work, regardless of the build backend in use!

Re: Use `Python -m Pip`

#50

Post by nl »

bxparks wrote: post #29

What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch easily among the various Python versions? Every time I try to research this, I get lost in the chaos of Python packaging and environments. Currently I do a `pip3 install -e .`, which uses the default Python provided by the OS. Then I hope that my continuous i…

If you scroll down in the linked article it says:

> ALWAYS use an environment! Don't install into your global interpreter!

...

> When you need to create an environment for a project I personally always reach for venv and virtual environments. It's included in Python's stdlib so it's always available via python -m venv (as long as you are not on Debian/Ubuntu, otherwise you may have to install the python3-venv apt package)

I think this is good advice, and it's pretty easy to do, and you can have named environments (via directory names) for each version you can switch between easily.

Very occasionally some python libraries will use a globally installed system library or a particular version which changes per python version (I think maybe OpenCV has this problem on Windows or something?) and so then you have to reach for containers.

Post reply on HN