Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to improve Python packaging

chriswarrick.com

Post reply on HN
Print view View original

141–150 of 204 posts

Re: How to improve Python packaging

#141

Post by NullInvictus »

bageljr wrote: post #137

I think it's crazy how they don't want to change cause it'd disrespect the work of maintainers of existing projects. Python packaging is screwed if maintaining all packaging software at the cost of functionality is the goal

At a previous company I had a chance to sit down and talk with python developers across a number of groups, and a fun question was asked: Imagine Python 4 is another large breaking change on the scale of 2 to 3. What feature makes it worth it to you?

The most common and enthusiastic answer was: Packaging and distribution.

I think to say that "large numbers of python developers think packaging is bad and should be fixed" undersells the current sentiment. I think there are large numbers of python developers with boots on the ground who are _utterly sick to death_ of the problem and willing to accept drastic measures resulting inconsiderable suffering if just means it means the issue goes away. People are in other words, getting rather desperate and discontent.

Upstream needs to do a little soul searching and come to an understanding that whatever ideological rock they're chained to that is preventing some solution from emerging has a high chance of not being moored in the needs of reality.

Re: How to improve Python packaging

#142

Post by alwayslikethis »

DangitBobby wrote: post #86
Lyngbakr wrote: post #21

It seems most people agree that Python's packaging isn't great, but, conversely, is there a language where most people agree that the approach to packaging is awesome? I mean, what's the gold standard to aspire to?

I think this is what success for unification looks like: Let's say there are two tools, one called pyup and one called pygo. pyup: responsible for which versions of python are available on your machine, as well as which one is the default "system" python. It also keeps itself and pygo updated. pygo: responsible for (among other things) - allowing a user to specify a python version for a project - allowing a user to s…

Gentoo's system sort of does both of these things. Among other things, you can control which packages are installed for which Python versions, dependencies across versions and default version for the system. Though it doesn't use the PyPI ecosystem directly, so everything has to be individually packaged for it to work.

Re: How to improve Python packaging

#143

Post by bageljr »

NullInvictus wrote: post #141
bageljr wrote: post #137

I think it's crazy how they don't want to change cause it'd disrespect the work of maintainers of existing projects. Python packaging is screwed if maintaining all packaging software at the cost of functionality is the goal

At a previous company I had a chance to sit down and talk with python developers across a number of groups, and a fun question was asked: Imagine Python 4 is another large breaking change on the scale of 2 to 3. What feature makes it worth it to you? The most common and enthusiastic answer was: Packaging and distribution. I think to say that "large numbers of python developers think packaging is bad and should be fix…

Well who is upstream? Pypa? Python steering council? Python itself won't do much without pypa approval. Pypa won't endorse an existing or new tool that solves most mainstream cases well.

Re: How to improve Python packaging

#144

Post by whatever1 »

Please allow me to vent on python packaging.

I was working to debug a very weird error that was happening in my code only when my app was raising a warning through my `warnings.py`.

My code was importing numpy, which in turn was trying to import its own `warnings.py`, but because of the python environment precedence, it was loading my own `warnings.py` which unfortunately had the same name.

How on earth was this something that passed the design of the import / package system of python? Why there are not unique id’s for the imported modules?

TLDR: An imported package took dependency on my own code because of module name clashing and lead to unexpected behavior.

Re: How to improve Python packaging

#145

Post by qbasic_forever »

theacodes wrote: post #138
KyeRussell wrote: post #129

Earlier quoted context omitted.

Yep. There being no documentation to really empower someone to work out what to use, to me very directly says “there’s some political spat going on that I’m being exposed to here”.

that "political spat" was actually targeted harassment towards a former PyPA member the last time someone actively working on packaging.python.org tried to be remotely opinionated. Since this ended with them stepping away and no one else has volunteered, it remains unopinionated.

Sounds like the steering council or whatever that runs python nowadays needs to step up. If people are being targeted for trying to improve packaging then the steering council should make a decision and direct all commentary, etc. to them, not individuals documenting best practices.

The whole 'let the community figure it out' seems to have failed and is causing nothing but more confusion and now attacks on people. The council needs to step up and say, "this is how python packaging will work, period. End of story, end of debate. There is no more discussion on this, the decision is final. All other python packaging tools now are non-standard and not recommended for use anymore".

That was the one good thing a BDFL model for leadership could achieve, making a hard decision in the face of many strong opinions.

Re: How to improve Python packaging

#146

Post by actinium226 »

Glad to see this getting discussion. I hate virtualenvs for several of the reasons mentioned in the article - extra state to keep track of, and potentially installing packages to the wrong location if I failed to keep track of it.

I always just force my user packages to work. I usually only use numpy/scipy/matplotlib and occasionally a few others so its not that hard, but some sort of npm like experience would be welcome. I know that many are struggling with these environments.

Re: How to improve Python packaging

#147

Post by vladvasiliu »

fbdab103 wrote: post #107
woodruffw wrote: post #55

Earlier quoted context omitted.

Oh, that's news to me! I stand corrected. (The point about other development tooling is, I believe, still accurate -- if you e.g. have `black` installed, `pip freeze` will show it.)

Which is a huge limitation of many of the other tools. I have some beef with poetry, but it did at least get one thing correct: differentiating between the code required libraries and the development tooling (pytest, black, etc). There are hacky workarounds to accomplish this with other tools, but codifying this differentiation is incredibly valuable.

With pip-tools you can use a requirements.txt for production and a requirements-dev.txt for your development environment. The latter imports the former.

Re: How to improve Python packaging

#148

Post by randomluck040 »

actinium226 wrote: post #146

Glad to see this getting discussion. I hate virtualenvs for several of the reasons mentioned in the article - extra state to keep track of, and potentially installing packages to the wrong location if I failed to keep track of it. I always just force my user packages to work. I usually only use numpy/scipy/matplotlib and occasionally a few others so its not that hard, but some sort of npm like experience would be wel…

As someone who has next to no experience with npm, which benefits of npm would you like to see in venv or package management in Python?

Re: How to improve Python packaging

#149

Post by vbernat »

groestl wrote: post #98
LtWorf wrote: post #56

Earlier quoted context omitted.

.deb packages…

Do you happen to have a link to a python application that ships it's own dependencies and has state-of-the art debian packaging?

It depends what you mean by state-of-the-art, but it's possible with dh-virtualenv: https://github.com/vincentbernat/pragmatic-debian-packages/t... (not something that can become an official package as it goes against Debian policies)

Re: How to improve Python packaging

#150

Post by Helmut10001 »

woodruffw wrote: post #11

It's not very well documented, but the PyPA tools do provide a unified experience when used correctly. Here's a PyPA project (FD: one I work on) that uses a single pyproject.toml to handle all aspects of packaging (and most non-packaging tool configuration, to boot)[1]. With a single file like that, the only thing you need to do to start a local development environment is: python -m venv env && . env/bin/activate pyt…

First time I hear PyPA, and looking up their website [1]: It is not a good sign if a packaging authority has a broken logo link on their mainpage [2].

[1]: https://www.pypa.io/ [2]: https://pypi.org/static/images/logo-large.svg/

Post reply on HN