Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to improve Python packaging

chriswarrick.com

Post reply on HN
Print view View original

131–140 of 204 posts

Re: How to improve Python packaging

#131

Post by luhn »

jasonhansel wrote: post #14

Over time, I've grown to appreciate "pip-tools." Since it's a dead-simple extension of pip, I wish it could be upstreamed into pip itself; that seems like the most straightforward way of fixing a number of Python's packaging issues.

I think a lot of people will wrinkle their nose at pip-tools because it's a more manual workflow, but I really enjoy that aspect of it. Other package managers I've used are convenient at first, but at some point I end up fighting with them and they won't give ground. With pip-tools, at the end of the day I'm in charge.

Plus I really like that the end result is a requirements.txt that can be used with any plain Python+pip environment.

Re: How to improve Python packaging

#132

Post by saila »

DangitBobby wrote: post #78
saila wrote: post #75

Earlier quoted context omitted.

If you don't mind adding a pyproject.toml, you could use pipx[1] to install these scripts. The directory structure would look like this: / pyproject.toml .py The pyproject.toml would look like this (using poetry, but you could use a different tool for this): [tool.poetry] # NOTE: Set to your project name name = " " description = "My Script" version = "0.0.0" authors = ["Me "] [tool.poetry.dependencies] python = "^3.1…

Thank you for trying to provide a workable solution, it's really not bad, but it has some downsides for me. pipx itself is installed inside a python environment, so when brew breaks my pythons, it breaks pipx as well. Anytime brew breaks my pythons, I would need to do the install step again for every script (or write a tool myself which does it). Not a total deal breaker, but not really much better than my current si…

I use `pip install --user pipx` to install pipx, but I think a better option when using homebrew would be `brew install pipx`. With the latter, doing a `brew upgrade` should keep python and pipx in step.

EDIT: Thinking about this some more, doing `brew install pipx` would keep pipx from breaking on brew upgrades, but I guess your installed scripts would still fail if you upgrade from Python 3.N to 3.N+1.

So the only solution, I think, is to keep around all the versions of Python you use, assuming you don't want to upgrade all your scripts when upgrading Python.

I use pyenv to manage Python versions rather than Homebrew, and this seems to avoid the issues you're having, because the Python version used to install whichever tool via pipx sticks around until I explicitly uninstall it.

Re: How to improve Python packaging

#133

Post by vonseel »

Today I was preparing a minimal project for something I needed to share with someone else. I wasn’t sure what tools they would already have installed, so I checked out Python’s official packaging docs and browsed around.

One page defaulted to giving instructions by default using “hatch”, while another page says the official recommendation is setuptools with a setup.cfg and only dynamic things declared in setup.py. Meanwhile, pyproject.toml support is being pushed elsewhere and in beta support for a lot of setup tools features.

There’s way too many tools and too much confusion around which ones to use and which ones are the best to choose going forward. Why can’t we just be like Rust and have one tool that builds and formats and runs tests and everything else?

Pipenv is crap but it’s somehow gained support of the PyCQA, meanwhile I’m not sure if poetry is even mentioned by the Python docs or CQA but it’s the one I’ve been using and it seems great, but I don’t even know if I’ve made the right choice anymore by using that.

All of this fragmentation just leads to developer confusion, newbies and people who have been using the language 10+ years alike.

Re: How to improve Python packaging

#134

Post by jamesfinlayson »

delijati wrote: post #46
jasonhansel wrote: post #14

Over time, I've grown to appreciate "pip-tools." Since it's a dead-simple extension of pip, I wish it could be upstreamed into pip itself; that seems like the most straightforward way of fixing a number of Python's packaging issues.

I work on mutlirepos and i really dislike gits subtree, subrepo i use https://pypi.org/project/zc.buildout/ . Yes i know i can do [1]. But editing on multirepos at the same time i can only do with zc.buildout [2] Still not perfect but it does the job. [1] $ cat req.txt requests git+ssh://git@gitlab.com/foo/bar/amber.git@0.4.0 git+ssh://git@gitlab.com/foo/bar/rebam.git@0.4.0 [2] $ cat buildout.cfg [buildout] extends =…

Ah, I haven't used buildout in years (I remember using it a lot when working with Plone).

I used it for a personal project but gave up a few years ago as some piece of the puzzle seemed broken and abandoned (something hadn't been updated to use a newer version of TLS or something).

I liked buildout though - it was a good system with its bin directory.

Re: How to improve Python packaging

#135

Post by thetic »

gen220 wrote: post #118
odiroot wrote: post #63

Earlier quoted context omitted.

Poetry still doesn't seem to support PEP621, instead requiring its custom, vendor-specific shape of pyproject.toml.

Have they spoken about why they don't support it? Or plans to support it in the future? It seems like the shortest path to a universal package management tool for Python is to add this capability to Poetry, rather than to build something new.

https://github.com/python-poetry/roadmap/issues/3

Re: How to improve Python packaging

#136

Post by thinker5555 »

TigerTeamX wrote: post #114

I must admit I just gave up on all these tools. Instead I just do pip install -r requirements.txt -t .lib and I have import sys import os sys.path.append(f"{os.getcwd()}/.lib") in the top of my script. Some will tell me this is silly, but it just works. Rememer to at .lib to your .gitignore. Else you'll have lot of fun.

I tinkered with this a little, and it's interesting. But what about using pip list or pip freeze? They don't show the packages in the .lib directory. Do you just add them by hand to the requirements.txt?

Re: How to improve Python packaging

#137

Post by bageljr »

I think it's crazy how they don't want to change cause it'd disrespect the work of maintainers of existing projects. Python packaging is screwed if maintaining all packaging software at the cost of functionality is the goal

Re: How to improve Python packaging

#138

Post by theacodes »

KyeRussell wrote: post #129
qbasic_forever wrote: post #53

Earlier quoted context omitted.

No I'm looking at link 1 and specifically have an issue with the huge table of options for how to setup your pyproject.toml--it has tabs for hatchling, flit, pdm, etc. but _zero_ description of why I would want to use one of those tools. It's just more confusing, like you're being thrown a bag of parts and no description of what to build or how to do it. To be honest that entire pypa doc should be like two paragraphs…

Yep. There being no documentation to really empower someone to work out what to use, to me very directly says “there’s some political spat going on that I’m being exposed to here”.

that "political spat" was actually targeted harassment towards a former PyPA member the last time someone actively working on packaging.python.org tried to be remotely opinionated. Since this ended with them stepping away and no one else has volunteered, it remains unopinionated.

Re: How to improve Python packaging

#139

Post by woodruffw »

KyeRussell wrote: post #129
qbasic_forever wrote: post #53

Earlier quoted context omitted.

No I'm looking at link 1 and specifically have an issue with the huge table of options for how to setup your pyproject.toml--it has tabs for hatchling, flit, pdm, etc. but _zero_ description of why I would want to use one of those tools. It's just more confusing, like you're being thrown a bag of parts and no description of what to build or how to do it. To be honest that entire pypa doc should be like two paragraphs…

Yep. There being no documentation to really empower someone to work out what to use, to me very directly says “there’s some political spat going on that I’m being exposed to here”.

Edit: My sibling comment has another answer, which makes me believe I'm lacking context. So I've removed this comment, in the interest of not offering opinions outside of what I know.

Re: How to improve Python packaging

#140

Post by theacodes »

TigerTeamX wrote: post #114

I must admit I just gave up on all these tools. Instead I just do pip install -r requirements.txt -t .lib and I have import sys import os sys.path.append(f"{os.getcwd()}/.lib") in the top of my script. Some will tell me this is silly, but it just works. Rememer to at .lib to your .gitignore. Else you'll have lot of fun.

As the person who implemented the -t flag and knows the horrendously sharp edge cases that it fails on, all I can say is bless your heart.
Post reply on HN