Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use `Python -m Pip`

snarky.ca

Post reply on HN
Print view View original

101–110 of 141 posts

Re: Use `Python -m Pip`

#101

Post by gernb »

woodruffw wrote: post #6

This is great advice. To tack on: Python has had built-in virtualenv support since 3.3[1], meaning that you can do this: python -m venv env/ ...on any version of Python released in the last decade and get a reasonable virtual environment. To go one step further, you can also have `venv` automatically bring `pip` and `setuptools` to their latest versions: python -m venv --upgrade-deps env/ ...which you should almost a…

and how am I able to run

    python -m venv env/
if haven't already installed python?

with node I use nvm. No root need, have never run into a problem of something complaining that I don't have node installed at a system level. Is there an equivalent for python?

Re: Use `Python -m Pip`

#102

Post by Chiron1991 »

gernb wrote: post #101
woodruffw wrote: post #6

This is great advice. To tack on: Python has had built-in virtualenv support since 3.3[1], meaning that you can do this: python -m venv env/ ...on any version of Python released in the last decade and get a reasonable virtual environment. To go one step further, you can also have `venv` automatically bring `pip` and `setuptools` to their latest versions: python -m venv --upgrade-deps env/ ...which you should almost a…

and how am I able to run python -m venv env/ if haven't already installed python? with node I use nvm. No root need, have never run into a problem of something complaining that I don't have node installed at a system level. Is there an equivalent for python?

pyenv. same thing as nvm, but for Python.

Re: Use `Python -m Pip`

#103

Post by takeda »

gernb wrote: post #99
newuser4321 wrote: post #15

Earlier quoted context omitted.

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

Could you please link to your favorite way to install python at specific version and all dependencies without have to put anything in a global place. Ideally on MacOS (since it's what I'm on most). I don't have homebrew or macports since eventually those always lead to me getting a borked system since they want everything installed globally.

Not the GP, but I became a big fan of using poetry for managing python package dependencies.

For managing python itself and binary libraries I started using Nix package manager.

It allows to describe all dependencies via code, but with time that code became a boilerplate, so I created this: https://github.com/takeda/nix-cde

It works very well for me so far.

You do need to have Nix[1] installed, but hopefully that should be the only thing needed and everything you can just list in project.nix file (in the example in README.md you should have access to `dive` command even if you never installed it on your system). Which will make it available only for that project.

Here's also an example of a very simple project: [2]

[1] https://nixos.org/download.html#nix-install-macos

[2] https://github.com/takeda/nix-cde/tree/master/tools/aws_assu...

Re: Use `Python -m Pip`

#104

Post by Hackbraten »

gernb wrote: post #99
newuser4321 wrote: post #15

Earlier quoted context omitted.

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

Could you please link to your favorite way to install python at specific version and all dependencies without have to put anything in a global place. Ideally on MacOS (since it's what I'm on most). I don't have homebrew or macports since eventually those always lead to me getting a borked system since they want everything installed globally.

Not OP but anecdotally, this is how my workflow looks like:

1. Install pyenv to a central location of my choice, e. g. ~/.pyenv (this helps manage all those different Python versions piling up from all the isolated projects I have.)

2. Install pipenv as a stand-alone tool. Doesn’t matter which Python I’m using, I just want to have it in my PATH.

3. Now I’m ready to create fully isolated projects using pipenv. Not only does pipenv create a venv right inside my project directory for me, it also helps me manage and lock dependencies. It also talks to pyenv so it can fetch the per-project Python version for me; this is especially useful when collaborating with others on the same project; they can reliably check out the project without having to manage specific Python versions themselves.

YMMV but I’ve had a stable experience using this setup so far.

Re: Use `Python -m Pip`

#105

Post by Hackbraten »

woodruffw wrote: post #69
newuser4321 wrote: post #15

Earlier quoted context omitted.

I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…

The conflicting advice is a serious problem. I hope you'll forgive me for adding one additional piece of advice: for many Python packages, the only packaging metadata you need is `pyproject.toml`. You don't even need `setup.py` anymore, so long as you're using a build backend that supports editable installs with `pyproject.toml`. Here's an example of a Python package that does everything in `pyproject.toml`[1]. You s…

Thanks for the link!

Re: Use `Python -m Pip`

#106

Post by okasaki »

bloblaw wrote: post #46
xdfgh1112 wrote: post #37

Earlier quoted context omitted.

I'm relatively new to python. I use venv, pip and requirements.txt. It's dead simple. What am I missing?

The fact that the suggested solution in Python is to give every Python script a full copy of an entire specific Python runtime (via venv) is a mild annoyance as a design pattern... to me . Python scripting today requires shipping your development environment. Python is wonderful until you want to run that code on another machine. At that point, the target system has to venv their way into reproducing your environment…

Virtualenvs are no worse than static linking, or packaging dlls/sos, all very common practice (and even recommended).

Python package management is VERY easy.

Re: Use `Python -m Pip`

#107

Post by Hackbraten »

bornfreddy wrote: post #95
butwhywhyoh wrote: post #43

I mean, this is generally good advice, but this reads like an infomercial where they show someone struggling REALLY hard to boil water to make a pot of spaghetti, when we all know it's really not that hard. They try to make finding your pip executable sound difficult, and even more difficult would be to understand what interpreter its tied to. Except... > pip -V > # pip 19.0.3 from /usr/local/lib/python2.7/site-packa…

This. Also, the argument is a bit weird: - always use python -m pip - if in venv, it doesn't do any good, but do it anyway for consistency - you should also use venv So... Maybe just always use virtual env and then you can use just pip?

While it’s good advice to use a venv for development work, there are a couple of Python packages that you want to have installed globally, such as pip, pipenv or poetry. So I don’t see a contradiction here after all.

Re: Use `Python -m Pip`

#108

Post by bcbrown »

figbert wrote: post #38

If I'm understanding correctly, this basically just kicks the ball a little further down the road... You shouldn't use pip directly because you don't know which version is the one in your path. Ok: the same applies to the python command? Calling pip is version ambiguous, but so is calling python.

It may be an ambiguous version, but it'll be the same version as the repl you get when you type `python`, and it'll be the same version that'll run your script when you type `python script.py`.

Re: Use `Python -m Pip`

#109

Post by jeroenhd »

gammalost wrote: post #8

Python is a fun language but the ecosystem around it is horrible. It's a shame. I just want to pip install like I would a package manager

The problem is that the operating system packages itself often depends on some version of a dependency. You can perfectly pip install something globally, but globally installing a library means everything else on your system is affected too.

There are solutions to this (pipenv being a popular one, though sadly often not explained to Python newcomers) and there are alternatives for that as well. You probably don't actually want to install a package like a package manager, most of the time you want to install an application and the specific requirements for that application, which is exactly what pipenv is for.

Your global package manager and your global Python package manager don't agree on what version is "the latest stable version" of a package so you just can't expect global package installs to work like that. It's like trying to make pacman, apt, and dnf work on the same file tree: you can probably get it done, but the end result will grow unstable or unusable within days.

Re: Use `Python -m Pip`

#110

Post by jeroenhd »

gernb wrote: post #101
woodruffw wrote: post #6

This is great advice. To tack on: Python has had built-in virtualenv support since 3.3[1], meaning that you can do this: python -m venv env/ ...on any version of Python released in the last decade and get a reasonable virtual environment. To go one step further, you can also have `venv` automatically bring `pip` and `setuptools` to their latest versions: python -m venv --upgrade-deps env/ ...which you should almost a…

and how am I able to run python -m venv env/ if haven't already installed python? with node I use nvm. No root need, have never run into a problem of something complaining that I don't have node installed at a system level. Is there an equivalent for python?

How am I able to run npm install/nvm install if I haven't already installed Node? I've seen way more machines that come with Python than I've seen machines that come with Node. You need to install and configure nvm, just like you need to install and configure Python.

The difference is that pip comes from a time when package managers were still universally global, while npm and friends only operate on directories by default.

If you want a systemless Python then you'll have to do it manually. I don't think it happens often enough for it to have a tool like nvm. Nothing prevents you from downloading the binaries, stuffing them into your user folder and setting up the right environment variables in your .profile, except for that it's a pain and probably not worth the effort.

Edit: apparently pyenv is a thing. I haven't ever needed it myself, but it seems to do everything nvm does and more.

If you just want to restrict package installs, versions, and configurations to a single project, virtual envs are the way to go.

Post reply on HN