Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on the Python packaging ecosystem

pradyunsg.me

Post reply on HN
Print view View original

1–10 of 121 posts

Re: Thoughts on the Python packaging ecosystem

#2

Post by DougMerritt »

At the top is a "tl;dr" which makes some points that I think are good to keep in mind for any set of tools:

   Pick from N different tools that do N different things is a good model.

   Pick from N ~equivalent choices is a really bad user experience.

   Picking a default doesn’t make other approaches illegal.

Re: Thoughts on the Python packaging ecosystem

#3

Post by zdragnar »

DougMerritt wrote: post #2

At the top is a "tl;dr" which makes some points that I think are good to keep in mind for any set of tools: Pick from N different tools that do N different things is a good model. Pick from N ~equivalent choices is a really bad user experience. Picking a default doesn’t make other approaches illegal.

As someone who has had to take over the projects of other people, I will say the third point needs a caveat:

    Picking a non-default tool needs a written explanation with sound reasoning
When there is a default, new members to the community will gravitate towards it and, if there isn't a good reason, will avoid learning other options.

In time, this ends up being anywhere from a minor annoyance to very frustrating, as not only are you losing time to learning, but potentially losing time to not having access to extras as the community around the default expands and your now special snowflake project goes without.

For very specific tools and libraries, it might be okay if they are easily interchangeable (but is apparently bad UX?) But for things with larger responsibilities - a full blown ORM, package manager, etc- it can end up being a millstone.

Re: Thoughts on the Python packaging ecosystem

#5

Post by chaxor »

Packaging and Nvidia are pretty much the main reasons python has degraded in status for the past 7 years. It started with the added and unnecessary confusion from conda (and silly things like pythonxy), but has really escalated through extra stupidities like poetry. Much of the features that may be enticing for these extra packages should have pushed the developers to improve the standard tools in python (pip). The other problem is due to python's lead in ML and the need to work with GPUs. That started with tensorflow, which had immense (still does) issues in dealing with versioning, mostly due to Nvidia (matching cuda versions and such). So effectively, Nvidia and package management killed a good language.

Re: Thoughts on the Python packaging ecosystem

#6

Post by woodruffw »

This is a great writeup by a central figure in Python packaging, and gets to the core of one of Python packaging's biggest strengths (and weaknesses): the PyPA is primarily an "open tent," with mostly independent (but somewhat standards-driven) development within it.

Pradyun's point about unnecessary competition rings especially true to me, and points to (IMO) a hard reality about where the ecosystem needs to go: at some point, there need to be some prescriptions about the one good tool to use for 99.9% of use cases, and with that will probably come some hurt feelings and disregarded technical opinions (including possibly mine!). But that's what needs to happen in order to produce a uniform tooling environment and UX.

Re: Thoughts on the Python packaging ecosystem

#7

Post by kelsolaar »

chaxor wrote: post #5

Packaging and Nvidia are pretty much the main reasons python has degraded in status for the past 7 years. It started with the added and unnecessary confusion from conda (and silly things like pythonxy), but has really escalated through extra stupidities like poetry. Much of the features that may be enticing for these extra packages should have pushed the developers to improve the standard tools in python (pip). The o…

Poetry certainly made our life much easier and was a productivity improvement. It the the tool that brought us back from Conda because it was able to solve environments than nothing in Pip land was able to do.

Re: Thoughts on the Python packaging ecosystem

#8

Post by saghm »

woodruffw wrote: post #6

This is a great writeup by a central figure in Python packaging, and gets to the core of one of Python packaging's biggest strengths (and weaknesses): the PyPA is primarily an "open tent," with mostly independent (but somewhat standards-driven) development within it. Pradyun's point about unnecessary competition rings especially true to me, and points to (IMO) a hard reality about where the ecosystem needs to go: at…

> Pradyun's point about unnecessary competition rings especially true to me, and points to (IMO) a hard reality about where the ecosystem needs to go: at some point, there need to be some prescriptions about the one good tool to use for 99.9% of use cases, and with that will probably come some hurt feelings and disregarded technical opinions (including possibly mine!). But that's what needs to happen in order to produce a uniform tooling environment and UX.

Not to mention that all efforts would be focused on improving a single package manager. Even if the one that was standardized wasn't the "best" at first (by whichever metric you prefer), giving everyone an incentive to improve that one will likely make it the best within a few years.

Re: Thoughts on the Python packaging ecosystem

#9

Post by jvolkman »

kelsolaar wrote: post #7
chaxor wrote: post #5

Packaging and Nvidia are pretty much the main reasons python has degraded in status for the past 7 years. It started with the added and unnecessary confusion from conda (and silly things like pythonxy), but has really escalated through extra stupidities like poetry. Much of the features that may be enticing for these extra packages should have pushed the developers to improve the standard tools in python (pip). The o…

Poetry certainly made our life much easier and was a productivity improvement. It the the tool that brought us back from Conda because it was able to solve environments than nothing in Pip land was able to do.

Was that before or after pip's new resolver was released at the end of 2020?

Re: Thoughts on the Python packaging ecosystem

#10

Post by atomicnumber3 »

Why does Ruby not have similar problems? (Or if it does, why does nobody seem to care?)

Like why does Ruby have gem and bundler, each doing one thing, whereas python has fifty bazillion tools that all do nearly the same thing if you squint but all have their own weird problems?

I've personally just ended up using poetry and that has mostly stopped me from having to care overmuch about the tooling.

I feel like I'm spoiled coming from Java land, where aside from too much XML, maven just works and if you need to do weird shit (like if you're android, for instance) then you use gradle, which still doesn't fuck up the maven repository format and we can all just live happily. (And nobody uses Ant anymore).

Post reply on HN