How to improve Python packaging

It's not very well documented, but the PyPA tools do provide a unified experience when used correctly.

Here's a PyPA project (FD: one I work on) that uses a single pyproject.toml to handle all aspects of packaging (and most non-packaging tool configuration, to boot)[1]. With a single file like that, the only thing you need to do to start a local development environment is:

    python -m venv env && . env/bin/activate
    python -m pip install .[dev]

(We provide a `make dev` target that does that for you as well.)

Similarly, to build distributions, all you need is `build`:

    python -m build

[1]: https://github.com/pypa/pip-audit

This post is prompted by the survey of Python users, their feedback, and a current thread [0] on the Python forms discussing a way forward. I read the thread the other day, it's long, and there are a lot of opinions.

Towards the end of this post there is an interesting observation:

> Discourse, the platform that the discussion was held on, shows the number of times a link was clicked. Granted, this count might not be always accurate, but if we assume it is, the link to the results summary was clicked only 14 times (as of 2023-01-14 21:20 UTC). The discussion has 28 participants and 2.2k views. If we believe the link click counter, half of the discussion participants did not even bother reading what the people think.

Along with that, my concern is that there are too my cooks in the kitchen. A packaging system is never going to cater for more than 90% of use cases, but there will likely be a higher than average representation from people who would sit in the 10%. There is a danger of them never being able to agree on a solution that solves all problems, and then never launch anything.

It would be far better to get something out, have a stake in the ground, and build from there. It also desperately needs to be "Core Python", my understanding is that the PyPA (Python Package Authority) is somewhat disconnected from the core Python development.

Ultimately they need one system that is front and centre, that is the "officially endorsed" tool, and very clear messaging on that.

0: https://discuss.python.org/t/python-packaging-strategy-discu...

Nowhere in pypa documentation is your simple workflow described or mentioned. Instead it's a jumble of links to a myriad of tools including hatchling, flit, pdm, etc. and basically just a shoulder shrug, 'I don't know, figure it all out yourself' message. This article makes a great point that the current pypa 'guidance' is too confusing and vague for actual end users (i.e. people that don't work directly on pypa or haven't been studying python packaging for decades).

Thanks, this is an interesting nugget. Did you intend to append a link reference for [0]?

I’m biased, but I do like Python’s package management much better than Node. Even just regular old virtualenvs. Can’t tell you how many times deleting the node packages dir and reinstalling fixes a weird issue, but that’s happened very rarely for me on the python side.

Also, having to comb through a bunch of node packages of dubious quality to find some sort of standard approach happens way too often. Like take python requests vs axios.