I use sites as composable mini venvs, but there's no tooling for that
How to improve Python packaging
41–50 of 204 posts
Re: How to improve Python packaging
#42Earlier quoted context omitted.
That repository doesn't seem to pin dependency versions. How do you integrate that in this workflow?
You could use `pip-compile` if you want full pinning. That's what we do on another project -- we use GitHub Actions with `pip-compile` to provide a fully frozen copy of the dependency tree for users who'd like that[1]. In the context of `pip-audit`, that makes a little less sense: most of our dependencies are semantically versioned, and we'd rather users receive patches and fixes to our subdependencies automatically,…
Or if you don't want to install something else and are willing to just use version numbers (instead of also hashes like pip-compile in that link), "pip freeze" is built in.
Re: How to improve Python packaging
#43It's interesting to note that PEP 582 (i.e. __pypackages__) mode used to be the PDM default -- and was sort of its flagship feature -- before being made opt-in with 2.0 release, the explanation being that the PEP had stalled, and that editor and IDE support for virtualenvs is much better (https://www.pythonbynight.com/blog/using-pdm-for-your-next-p...).
If you read through the discussion on the Python forum, one of the counterarguments to the PEP is that "If existing tools can already support this, why do we need a PEP for it?" But presumedly a PEP would help push the ecosystem to accommodate __pypackages__, and to solve the aforementioned problems (like broader editor and IDE support).
For what it's worth (as someone that builds Python tooling full-time): I'm generally a fan of moving to a node_modules-like model.
Re: How to improve Python packaging
#44It's interesting to note that PEP 582 (i.e. __pypackages__) mode used to be the PDM default -- and was sort of its flagship feature -- before being made opt-in with 2.0 release, the explanation being that the PEP had stalled, and that editor and IDE support for virtualenvs is much better ( https://www.pythonbynight.com/blog/using-pdm-for-your-next-p... ). If you read through the discussion on the Python forum, one of…
Oh, also, I think I've read the entire Discussions thread on this PEP, and from what I can tell, the PEP is missing champions who are eager to push it forward.
Part of the problem, I'm guessing, is that the PEP kind of sits in a weird space, because it's not up to the PyPA to "approve" it, or whatever -- PEPs are approved by the Python Steering Council.
Per Brett Cannon (Steering Council Member):
> Either consensus has to be reached (which it hasn’t) and/or someone needs to send this to the SC to make a decision (although it is a weird PEP in that it also impacts packaging, so it isn’t even clear who would get final say).
https://discuss.python.org/t/pep-582-python-local-packages-d...
Re: How to improve Python packaging
#45It's interesting to note that PEP 582 (i.e. __pypackages__) mode used to be the PDM default -- and was sort of its flagship feature -- before being made opt-in with 2.0 release, the explanation being that the PEP had stalled, and that editor and IDE support for virtualenvs is much better ( https://www.pythonbynight.com/blog/using-pdm-for-your-next-p... ). If you read through the discussion on the Python forum, one of…
That tooling argument is quite weak. PDM is the only tool I was able to find that has support for __pypackages__, the paths it uses seem to be slightly different to the PEP wording, and it uses a $PYTHONPATH/shell hack (`pdm --pep582` adds a folder with `sitecustomize.py` that does the magic). How do you change the $PYTHONPATH used by an IDE (and its magic features)?
Re: How to improve Python packaging
#46Over time, I've grown to appreciate "pip-tools." Since it's a dead-simple extension of pip, I wish it could be upstreamed into pip itself; that seems like the most straightforward way of fixing a number of Python's packaging issues.
I work on mutlirepos and i really dislike gits subtree, subrepo i use https://pypi.org/project/zc.buildout/. Yes i know i can do [1]. But editing on multirepos at the same time i can only do with zc.buildout [2] Still not perfect but it does the job.
[1]
$ cat req.txt
requests
git+ssh://git@gitlab.com/foo/bar/amber.git@0.4.0
git+ssh://git@gitlab.com/foo/bar/rebam.git@0.4.0
$ cat buildout.cfg
[buildout]
extends = versions.cfg
extensions = mr.developer
auto-checkout = \*
develop = .
show-picked-versions = true
update-versions-file = versions.cfg
sources-dir = git-sources
parts = py
eggs =
amber
rebam
hammer
[sources]
amber = git git@gitlab.com/foo/bar/amber.git@0.4.0
rebma = git git@gitlab.com/foo/bar/rebma.git@0.4.0
[py]
recipe = zc.recipe.egg
eggs =
${buildout:eggs}
interpreter = py-backend
dependent-scripts = trueRe: How to improve Python packaging
#47Cant resist digging at Node.js even when writing up how infinitely better Node.js is at dealing with packages than python, haha: > Let’s try removing is-odd to demonstrate how badly designed this package is: You literally just deleted is-even 's dependency on is-odd then have the audacity to be shocked that it broke? There's a lot of hatred for the small package philosophy of node.js, but it's also a huge win, stands…
There’s a place for small packages, but is-even/is-odd is a bit too small to be a reasonable package. It is far easier to just write `x % 2 === 0` inline, which is an obvious idiom, instead of installing and importing a separate package for this. The use of is-odd by is-even can be confusing for users. For example, you may call isEven(0.5) and get the following error: RangeError: is-odd expects an integer. at isOdd (…
isEven is in thay stack trace - should not confuse anyone with even a basic introductory level fluency at coding.
Is it too small? What if latter the language evolves BigInt? Donwe suffer a patchwork of libraries which have & havent upgraded, sussing around each time to find out?
I think the key thing to recognize is that this is all opinion. Many people dont like the availability of many opions, the ease at which dependencies have grown. And that's fine, there's some real pain here to having ballooning package trees. There's a pevel of conceit though that I feel that often arises, where we mock & shiv packages like is-even . But to me, it's not absolute, it's a matter of taste & preference. It looks weird to outsiders, but it has been enormously powerful & helpful, has been such a key successful element of JS that npm arose & made package management easy & package publishing easy & that we begat a new behavior of capturing all the little helpful things we do & making them available.
Maybe there are good reasons for inlining simple things, but it's not clear to me what the gains really are, or what's wrong with is-even.
Re: How to improve Python packaging
#48One more thing we need is app distribution for desktop apps written in python. I need to be able to package python app as single binary. There are tools like pyoxidizer https://pyoxidizer.readthedocs.io/en/stable/ . hopefully it become standard in python community
You probably already know about this, but Nuitka[0] is pretty great for building distributable Python apps.
[0]: https://nuitka.net
Re: How to improve Python packaging
#49It's interesting to note that PEP 582 (i.e. __pypackages__) mode used to be the PDM default -- and was sort of its flagship feature -- before being made opt-in with 2.0 release, the explanation being that the PEP had stalled, and that editor and IDE support for virtualenvs is much better ( https://www.pythonbynight.com/blog/using-pdm-for-your-next-p... ). If you read through the discussion on the Python forum, one of…
That tooling argument is quite weak. PDM is the only tool I was able to find that has support for __pypackages__, the paths it uses seem to be slightly different to the PEP wording, and it uses a $PYTHONPATH/shell hack (`pdm --pep582` adds a folder with `sitecustomize.py` that does the magic). How do you change the $PYTHONPATH used by an IDE (and its magic features)?
I agree!
Re: How to improve Python packaging
#50Earlier quoted context omitted.
You could use `pip-compile` if you want full pinning. That's what we do on another project -- we use GitHub Actions with `pip-compile` to provide a fully frozen copy of the dependency tree for users who'd like that[1]. In the context of `pip-audit`, that makes a little less sense: most of our dependencies are semantically versioned, and we'd rather users receive patches and fixes to our subdependencies automatically,…
Or if you don't want to install something else and are willing to just use version numbers (instead of also hashes like pip-compile in that link), "pip freeze" is built in.
The tricky thing with `pip freeze` is that it dumps your environment, not your resolved set: your environment also contains things like your `pip` and `setuptools` versions, any developing tooling you have, and potentially your global environmental state (if the environment has global access and you forget to pass `--local` to `pip freeze`).
In other words, it's generally a superset of the resolutions collected by `pip-compile`. This may or may not be what you want, or what your users expect!