Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on the Python packaging ecosystem

pradyunsg.me

Post reply on HN
Print view View original

101–110 of 121 posts

Re: Thoughts on the Python packaging ecosystem

#101

Post by LtWorf »

Hackbraten wrote: post #92
LtWorf wrote: post #44

Earlier quoted context omitted.

If you have multiple versions of python installed you can just do python3.9 -m pip python3.10 -m pip The standard library is the reason why python is popular to begin with.

To me, that’d be a UX nightmare. Even if I tried to, I’m not going to memorize which of my 20 checked-out Python projects are requiring which Python invocation. IMHO it’s absolutely the toolchain’s job to manage that for me. I’d never adopt a toolchain that wouldn’t.

You do know that python is backwards compatible right?

You can just use 3.11 with everything.

Re: Thoughts on the Python packaging ecosystem

#102

Post by ehutch79 »

rerdavies wrote: post #88
ehutch79 wrote: post #61

Earlier quoted context omitted.

There’s nothing really wrong with pip. It doesn’t have a lot of features, but its position isn’t one to have those features. It’s simple ina good way. I don’t need to futz with a bunch of stuff I don’t need to get it going

Keras/tensorflow has some version installed, pip install fails. Being able to install multiple versions of the same module seems like a fairly basic feature for a packaging manager. And it would seem to me that having a shim executable to examine a profile of some kind for an app at startup, and load/install the correct version of python doesn't seem impossible.

I find having multiple versions weird. That’s gotta be a niche need.

If nothing else it seems like a good way to let packages be lax in keeping their dependencies up to date.

Re: Thoughts on the Python packaging ecosystem

#103

Post by Kwpolska »

ivoflipse wrote: post #50
pronlover723 wrote: post #49

For me, before I even get to packaging, I hit the install/environment issue with python. Python, by default, wants to be installed at a system level and wants libraries/packages to be at a system level. That shit has to stop. The default needs to be project local installs. Node might have issues but one thing they got right is defaulting to project local installs vs python where you need various incantations to get o…

Lucky for you, there's a new PEP proposing just that: https://discuss.python.org/t/pep-704-require-virtual-environ... Written by the author of this blog post no less! However, you'll find that as with all packaging discussions there are people opposing it, because their workflow doesn't match yours and they don't want to change how they work. We, as a community, need a way to resolve such stalemates or I fear we won'…

There’s a competing and saner PEP that actually proposes local package directories (which are not venvs): https://discuss.python.org/t/pep-582-python-local-packages-d...

Re: Thoughts on the Python packaging ecosystem

#104

Post by lmm »

LtWorf wrote: post #101
Hackbraten wrote: post #92

Earlier quoted context omitted.

To me, that’d be a UX nightmare. Even if I tried to, I’m not going to memorize which of my 20 checked-out Python projects are requiring which Python invocation. IMHO it’s absolutely the toolchain’s job to manage that for me. I’d never adopt a toolchain that wouldn’t.

You do know that python is backwards compatible right? You can just use 3.11 with everything.

Traditionally it wasn't, and even a "minor" upgrade was a significant undertaking for a large project. Maybe that's changed.

Re: Thoughts on the Python packaging ecosystem

#105

Post by adammarples »

AlphaSite wrote: post #63
divbzero wrote: post #26

Earlier quoted context omitted.

To me the natural choice would be pip which, as OP points out, has the advantage of shipping with Python by default. We could remedy the shortcomings of pip by incorporating the most successful features from the N alternatives that exist today. The npm / yarn fork that formed in 2016 is the closest analogy I can think of. My impression is that npm improved on its biggest weaknesses ( e.g. a lock file) and has remaine…

Even if pip was a little better I still wouldn’t use it, because Poetry solves the problem so well you’d need to pull it out of my cold hands. It solves not only the install package problem, but locking, venvs, project structure and packaging in a well integrated solution with a fantastic UX. If poetry (or similar) were renamed to Pip and it were to be included in the stdlib id obviously switch. It doesn’t solve the…

I've given up on poetry. I still to this day can't get it to use the active version of python, which you'd think would be the default. Even after setting the config flag. Setting the config flag somehow made things worse, all my poetry envs started using the wrong python. I can't even force it to use the version I want with 'env use /path/to/python', I can only pass stuff like 'env use 3.9'. What the hell is 3.9? I want to pass a specific path to a version I have.

Re: Thoughts on the Python packaging ecosystem

#106

Post by mattbillenstein »

lmm wrote: post #104
LtWorf wrote: post #101

Earlier quoted context omitted.

You do know that python is backwards compatible right? You can just use 3.11 with everything.

Traditionally it wasn't, and even a "minor" upgrade was a significant undertaking for a large project. Maybe that's changed.

Maybe in the 2.4 -> 2.7 transitions - Python3 after about 3.6 is pretty good, you might see some minor package breakage if you're running the very latest version, but that's fairly rare.

Generally speaking, if you're on a modern OS - like recent Ubuntu LTS or MacOS you're getting 3.8 -> 3.10 and compatibility is very good in these releases.

Re: Thoughts on the Python packaging ecosystem

#107

Post by acdha »

lmm wrote: post #104
LtWorf wrote: post #101

Earlier quoted context omitted.

You do know that python is backwards compatible right? You can just use 3.11 with everything.

Traditionally it wasn't, and even a "minor" upgrade was a significant undertaking for a large project. Maybe that's changed.

When exactly are you defining as “traditional”? The Python point releases have generally been painless for a long time. There was a little friction in the early 2-3 era before they made it easier to write code which worked on both without translation but in general it’s been at least a decade since I’ve cared about this except for one time where a couple of libraries depended on a private API which changed in Python 2.7.9, and that was back in 2014.

https://bugs.python.org/issue22438

Re: Thoughts on the Python packaging ecosystem

#108

Post by uranusjr »

numbsafari wrote: post #84
earthboundkid wrote: post #53

Earlier quoted context omitted.

We need some person, a nice person, a benevolent person, who could some how tell other people what to do, dictate it if you will, and it would be best if they could keep up this job for the rest of his or her life. We’ll call them, the Friendly Language Uncle.

I’d prefer a crazy aunt to a creepy uncle… but maybe let’s leave family out of it altogether.

Parent is referring to a long-running (but somewhat obscure at this point unfortunately) joke

https://peps.python.org/pep-0401/

Re: Thoughts on the Python packaging ecosystem

#109

Post by chucke »

atomicnumber3 wrote: post #10

Why does Ruby not have similar problems? (Or if it does, why does nobody seem to care?) Like why does Ruby have gem and bundler, each doing one thing, whereas python has fifty bazillion tools that all do nearly the same thing if you squint but all have their own weird problems? I've personally just ended up using poetry and that has mostly stopped me from having to care overmuch about the tooling. I feel like I'm spo…

Ruby was actually the precursor to the solutions seen nowadays. It was thre first mainstream case of a packaging tool using lockfiles, and one of the persons behind it was also involved in the development of yarn (which brought lockfiles to node) and rust cargo.

Re: Thoughts on the Python packaging ecosystem

#110

Post by thraxil »

pronlover723 wrote: post #77
asix66 wrote: post #54

Earlier quoted context omitted.

You can do this now by creating a Python virtual environment[0]. Then you can package your project with a requirements file and some instructions on its use. I use Python VENV often, and it works really well. [0] https://realpython.com/python-virtual-environments-a-primer/

Apparently you missed the point. local needs to be the default. Not some magic incantation In node cd projectFoo # do stuff, node uses packages local to projectFoo cd ../projectBar # do stuff, node uses packages local to projectBar There is no "activation", the default is it just works. Installing packages local to the project is also the default.

Activating virtualenvs isn't necessary. In every python project I work in, I do

    $ cd projectFoo
    $ ./ve/bin/python whatever...
(more realistically, it's `make whatever` which then builds the virtualenv into `./ve` if needed, pip installs required packages into it, and runs the command).

Yes, I agree that it would be nice if the default behaviour of `pip install -r requirements.txt` was to install it in an isolated virtualenv specific to that project, but it's not also not like it's completely impossible magic.

Post reply on HN