Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use `Python -m Pip`

snarky.ca

Post reply on HN
Print view View original

71–80 of 141 posts

Re: Use `Python -m Pip`

#71

Post by lovingCranberry »

newuser4321 wrote: post #15
gammalost wrote: post #8

Python is a fun language but the ecosystem around it is horrible. It's a shame. I just want to pip install like I would a package manager

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

I wish there was a "standard" package manager like npm. While npm might not be the most efficient one, it just works.

Re: Use `Python -m Pip`

#72

Post by na85 »

bloblaw wrote: post #46
xdfgh1112 wrote: post #37

Earlier quoted context omitted.

I'm relatively new to python. I use venv, pip and requirements.txt. It's dead simple. What am I missing?

The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern... to me . Python scripting today requires shipping your development environment. Python is wonderful until you want to run that code on another machine. At that point, the target system has to venv their way into reproducing your environment…

>The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern...to me.

It's not just a mild annoyance, it's a sad statement about python as a community of developers that this is not only accepted but recommended.

I write python because some of what I do occurs in the domains where python makes the most sense, but the idea that the python community accepts venv rather than considering the fact that venv even exists at all to be a source of profound embarassment is mystifying to me.

Re: Use `Python -m Pip`

#73

Post by romanows »

jjnoakes wrote: post #63
ericholscher wrote: post #62

Earlier quoted context omitted.

Isn’t that true of command execution as well? I guess you could use a full path to the pip executable, but similar issue.

Do you mean there might be a 'pip' command in the local directory that might get called by accident, or do you mean the real python pip command might load a python module from the local directory by accident?

The second thing. Python includes the current working directory first in the module search path. It leads to the above issue and occasionally some a tricky debugging problem.

Re: Use `Python -m Pip`

#74

Post by danudey »

jjnoakes wrote: post #63
ericholscher wrote: post #62

Earlier quoted context omitted.

Isn’t that true of command execution as well? I guess you could use a full path to the pip executable, but similar issue.

Do you mean there might be a 'pip' command in the local directory that might get called by accident, or do you mean the real python pip command might load a python module from the local directory by accident?

Windows' cmd.exe is the only shell I'm aware of that (by default) checks the local directory for executables before the actual PATH variable, so I wouldn't consider that a real problem.

The 'local directory' one is the actual concern. For example, our CI system for one of our tools runs commands like `pip install -U setuptools`. If we switched it to `python -m pip install -U setuptools`, it would continue to work fine - unless a developer accidentally committed a file called `pip.py` to the root.

At that point, Python would try to import `pip.py` as `pip` instead of the actual module pip (because current directory is checked for includes first).

Re: Use `Python -m Pip`

#75

Post by thaumasiotes »

singron wrote: post #64
ericholscher wrote: post #62

Earlier quoted context omitted.

Isn’t that true of command execution as well? I guess you could use a full path to the pip executable, but similar issue.

Only if PATH includes ".", which is why they say not to do that (or use windows. I'm not sure if you can fix that there).

Even if PATH includes the current directory, you'll be safe as long as it doesn't take priority over the regular bin directories.

Re: Use `Python -m Pip`

#76

Post by rat87 »

eslaught wrote: post #24
newuser4321 wrote: post #15

Earlier quoted context omitted.

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

To be honest, I don't see how you can give this advice. A couple of the most popular package management systems for Python are very obviously deficient (and sigh, these are usually the ones I get stuck working with, due to outside constraints). A classic example is version pinning. Rust has Cargo.lock, Ruby has Gemfile.lock. Python? It depends on which one of the multitude of options you pick. But at least a couple o…

python's poetry has poetry.lock I think it's slowly becoming semi-standard. But still slowly and still not fully ready(bugs). Also its odd that pyproject.toml (not poetry.toml which is the poetry settings for repo) is the dependency definition file and poetry.lock is the lock file

for pip file I usually make a requirements-to-freeze.txt, make a fresh virtualenv, install requirements-to-freeze.txt and then pip freeze > requirements.txt poetry does this automatically

Re: Use `Python -m Pip`

#77

Post by danudey »

andreareina wrote: post #59
bloblaw wrote: post #46

Earlier quoted context omitted.

The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern... to me . Python scripting today requires shipping your development environment. Python is wonderful until you want to run that code on another machine. At that point, the target system has to venv their way into reproducing your environment…

Ansible is a special example of this. Some Ansible modules (e.g. the postgresql one) require non-stdlib modules to be installed in the remote python. So either install it globally or figure out how to make Ansible work with environments. Even if the thing you're trying to set up ostensibly doesn't use python. sigh

As I recall, pipx does this; it installs a separate env, but then provides entrypoints (commands) to the global environment.

For example, it would install ansible and the ansible dependencies to a separate environment somewhere, but still put commands for `ansible`, `ansible-playbook`, etc. into /usr/local/bin for example. When you run the ansible command, it loads ansible from the separate environment.

I haven't actually tried this myself since our environments are pretty uniform, but it could be worth looking into. It would be nice if pip itself provided this as an option (or if it became the default), but that could also become extremely complicated in terms of upgrades. I already hate having to download and build scipy, but it would be incredibly irritating to have to do it once for every tool I have that uses it. I'm sure some kind of cache could be made to work, but it's not trivial.

Re: Use `Python -m Pip`

#78

Post by thaumasiotes »

bloblaw wrote: post #46
xdfgh1112 wrote: post #37

Earlier quoted context omitted.

I'm relatively new to python. I use venv, pip and requirements.txt. It's dead simple. What am I missing?

The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern... to me . Python scripting today requires shipping your development environment. Python is wonderful until you want to run that code on another machine. At that point, the target system has to venv their way into reproducing your environment…

> The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern...to me.

That bothers me too, a bit. But sidethread to you people are comparing python unfavorably to ruby, which as far as I know uses exactly the same strategy (except that it's called "rvm" instead of "virtualenv"). Node.js appears to do the same thing.

To my eyes, the solution to this would be static linking, but I don't know how much sense that makes in the python / ruby / js context. Other than that, what's the alternative? There's certainly a lot of convergence on this solution.

Re: Use `Python -m Pip`

#79

Post by returnzero »

rat87 wrote: post #76
eslaught wrote: post #24

Earlier quoted context omitted.

To be honest, I don't see how you can give this advice. A couple of the most popular package management systems for Python are very obviously deficient (and sigh, these are usually the ones I get stuck working with, due to outside constraints). A classic example is version pinning. Rust has Cargo.lock, Ruby has Gemfile.lock. Python? It depends on which one of the multitude of options you pick. But at least a couple o…

python's poetry has poetry.lock I think it's slowly becoming semi-standard. But still slowly and still not fully ready(bugs). Also its odd that pyproject.toml (not poetry.toml which is the poetry settings for repo) is the dependency definition file and poetry.lock is the lock file for pip file I usually make a requirements-to-freeze.txt, make a fresh virtualenv, install requirements-to-freeze.txt and then pip freeze…

> But still slowly and still not fully ready(bugs).

I use poetry every day and so far it's been pretty great. The main complications I have had are with accessing private pypi repos in Azure DevOps pipelines which use short lived tokens but it just looks a little clunky, still works.

> Also its odd that pyproject.toml (not poetry.toml which is the poetry settings for repo) is the dependency definition file and poetry.lock is the lock file

This is because poetry is using Python's PEP 518[1] specification rather than define their own build requirements format. It also isn't limited to just building, you can also include the configuration for other python tools like `pytest`[2].

[1] https://peps.python.org/pep-0518/

[2] https://docs.pytest.org/en/stable/reference/customize.html#p...

Re: Use `Python -m Pip`

#80

Post by rootusrootus »

eslaught wrote: post #24
newuser4321 wrote: post #15

Earlier quoted context omitted.

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

To be honest, I don't see how you can give this advice. A couple of the most popular package management systems for Python are very obviously deficient (and sigh, these are usually the ones I get stuck working with, due to outside constraints). A classic example is version pinning. Rust has Cargo.lock, Ruby has Gemfile.lock. Python? It depends on which one of the multitude of options you pick. But at least a couple o…

I must live a sheltered life. Everything I've run into so far uses requirements.txt to pin dependencies. Well supported and easy is probably why.
Post reply on HN