Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on the Python packaging ecosystem

pradyunsg.me

Post reply on HN
Print view View original

31–40 of 121 posts

Re: Thoughts on the Python packaging ecosystem

#31

Post by 0cf8612b2e1e »

atomicnumber3 wrote: post #10

Why does Ruby not have similar problems? (Or if it does, why does nobody seem to care?) Like why does Ruby have gem and bundler, each doing one thing, whereas python has fifty bazillion tools that all do nearly the same thing if you squint but all have their own weird problems? I've personally just ended up using poetry and that has mostly stopped me from having to care overmuch about the tooling. I feel like I'm spo…

Not a Ruby guy, but my feeling is that C extensions are more rare in that ecosystem. The majority of the Python woes feel directed at the proper way to package and compile non-Python code.

Re: Thoughts on the Python packaging ecosystem

#32

Post by lmm »

divbzero wrote: post #26
woodruffw wrote: post #6

This is a great writeup by a central figure in Python packaging, and gets to the core of one of Python packaging's biggest strengths (and weaknesses): the PyPA is primarily an "open tent," with mostly independent (but somewhat standards-driven) development within it. Pradyun's point about unnecessary competition rings especially true to me, and points to (IMO) a hard reality about where the ecosystem needs to go: at…

To me the natural choice would be pip which, as OP points out, has the advantage of shipping with Python by default. We could remedy the shortcomings of pip by incorporating the most successful features from the N alternatives that exist today. The npm / yarn fork that formed in 2016 is the closest analogy I can think of. My impression is that npm improved on its biggest weaknesses ( e.g. a lock file) and has remaine…

Shipping with python is a disadvantage; any real workflow tool needs to be able to manage multiple python installations, and if you try to use a part of the python installation to do that you've got a circular dependency problem. Plus the whole "where modules go to die" thing.

Re: Thoughts on the Python packaging ecosystem

#33

Post by musicale »

ram_rar wrote: post #15

I get a lot of flak for saying this, but I do hope python committers and psf members take a hard look at perl and its demise. Perl had a very similar problem of having N equivalent choices for doing the same thing and python for better or worse is heading in the same direction. In the last few yrs of my professional career many projects I worked, were on are migrating python -> golang cuz frankly many people were fed…

Python's aggressive deprecation schedule is a major disadvantage compared to languages with greater stability and backward compatibility.

Making things better for dozens of maintainers at the expense of millions of users is the wrong tradeoff.

Apple does something similar (e.g. with Swift and with iOS), offloading a constant support burden onto developers.

Re: Thoughts on the Python packaging ecosystem

#34

Post by pdimitar »

wheelerof4te wrote: post #24
chaxor wrote: post #5

Packaging and Nvidia are pretty much the main reasons python has degraded in status for the past 7 years. It started with the added and unnecessary confusion from conda (and silly things like pythonxy), but has really escalated through extra stupidities like poetry. Much of the features that may be enticing for these extra packages should have pushed the developers to improve the standard tools in python (pip). The o…

Python's demise would be it's undeserving fate to be the world's best "glue" language. The language needs to stand on it's own to be competitive in the long run. Pure Python modules are much easier to install and keep updated.

It will be a VERY deserving fate, since the community is always busy fighting amongst themselves and the maintainers are conservative to the point of being unreasonable.

Maybe such a community deserves obscurity.

Python has real problems that need addressing, and that was true 10 years ago as well, packaging included. Pretending this isn't the case makes more serious techies just laugh at Python, and it's fully deserved.

We do tech, for God's sake. We don't do religion. Merit is what should call the shots.

Re: Thoughts on the Python packaging ecosystem

#35

Post by Kamq »

wheelerof4te wrote: post #21

There really needs to be one tool that combines installing, building, testing and running Python projects. This tool should be officially endorsed by core developers of Python and gradually added to the standard distribution model. The good starting point for this would be Pip. It is de facto standard tool for installing packages. It can be extended to do more. Also, the standard should favor pure-Python packages, in…

> Also, the standard should favor pure-Python packages, in order to untangle Python from it's legacy as a glue language for modules written in C(++).

Never gonna happen. Python's explosion has been through data science and ml, which is based entirely around calling Fortran/C/C++/etc.

> Otherwise, no progress will be made at the language level and Python will die out once C becomes legacy language, replaced by safer systems programming languages.

I assume you mean Rust here. Do you think python can't already call rust? It's been able to do that for the better part of a decade.

I mean, this is once of python's main selling points. It doesn't matter what your numeric/scientific library is written in, python can call it, and it's vaguely convenient to write (depending on your taste for comprehensions).

Why would the python ecosystem shoot themselves in the foot like that?

Re: Thoughts on the Python packaging ecosystem

#36

Post by Aeolun »

divbzero wrote: post #26
woodruffw wrote: post #6

This is a great writeup by a central figure in Python packaging, and gets to the core of one of Python packaging's biggest strengths (and weaknesses): the PyPA is primarily an "open tent," with mostly independent (but somewhat standards-driven) development within it. Pradyun's point about unnecessary competition rings especially true to me, and points to (IMO) a hard reality about where the ecosystem needs to go: at…

To me the natural choice would be pip which, as OP points out, has the advantage of shipping with Python by default. We could remedy the shortcomings of pip by incorporating the most successful features from the N alternatives that exist today. The npm / yarn fork that formed in 2016 is the closest analogy I can think of. My impression is that npm improved on its biggest weaknesses ( e.g. a lock file) and has remaine…

To some extend. NPM is still (even after all the improvements) the weakest of the package managers.

There’s no doubt it’s the default though.

Re: Thoughts on the Python packaging ecosystem

#37

Post by Aeolun »

KyeRussell wrote: post #27
jessaustin wrote: post #17

Earlier quoted context omitted.

...aren't there at least two or three package managers (npm, yarn, maybe pnpm)? I'm not sure about yarn, but pnpm has exactly the same API as npm has. It simply has a different disk organization and caching strategy. If npm maintainers so chose, pnpm's behavior would be an option within npm. Since they haven't chosen that, it seems completely reasonable to "compete" in the way that pnpm does.

Yes, but one has to come to that conclusion oneself, which increases friction. I realise you’re talking about pnpm, and not yarn, but when I think of issues like this, yarn is what comes to mind. A bunch of projects prefer yarn over npm, or at the very least place them side by side, in the project documentation. As someone reasonably familiar with JavaScript, but that only works with it sporadically, I always find my…

I think the nice thing about the node ecosystem is that all the packaging tools are using the exact same package.json. The only thing that’s different is the way they resolve/install the packages.

Re: Thoughts on the Python packaging ecosystem

#38

Post by solarkraft »

wheelerof4te wrote: post #21

There really needs to be one tool that combines installing, building, testing and running Python projects. This tool should be officially endorsed by core developers of Python and gradually added to the standard distribution model. The good starting point for this would be Pip. It is de facto standard tool for installing packages. It can be extended to do more. Also, the standard should favor pure-Python packages, in…

I'm doubly confused. Not only is Python perfectly capable of binding to Rust, but also why would you want things you'd usually do in faster languages to be done in Python? It's great as glue but I think we'd agree that it's not very good at some performance critical things.

Re: Thoughts on the Python packaging ecosystem

#39

Post by solarkraft »

lmm wrote: post #32
divbzero wrote: post #26

Earlier quoted context omitted.

To me the natural choice would be pip which, as OP points out, has the advantage of shipping with Python by default. We could remedy the shortcomings of pip by incorporating the most successful features from the N alternatives that exist today. The npm / yarn fork that formed in 2016 is the closest analogy I can think of. My impression is that npm improved on its biggest weaknesses ( e.g. a lock file) and has remaine…

Shipping with python is a disadvantage; any real workflow tool needs to be able to manage multiple python installations, and if you try to use a part of the python installation to do that you've got a circular dependency problem. Plus the whole "where modules go to die" thing.

Does anything speak against flipping the relationship? i.e. using pip (or something else) as the standard way to install Python.

Re: Thoughts on the Python packaging ecosystem

#40

Post by solarkraft »

I don't know, I'm pretty happy with Pipenv. It's the standard among the people I talk to and very good at being "pip, but better". I've tried poetry but don't really find it good enough to justify being so different, maybe unless you already have a pyproject.toml anyway.

There's a lot of talk about Python packaging but as a semi-casual user I've actually had very few problems with it.

Post reply on HN