Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to improve Python packaging

chriswarrick.com

Post reply on HN
Print view View original

101–110 of 204 posts

Re: How to improve Python packaging

#101

Post by DangitBobby »

noisy_boy wrote: post #93
DangitBobby wrote: post #78

Earlier quoted context omitted.

Thank you for trying to provide a workable solution, it's really not bad, but it has some downsides for me. pipx itself is installed inside a python environment, so when brew breaks my pythons, it breaks pipx as well. Anytime brew breaks my pythons, I would need to do the install step again for every script (or write a tool myself which does it). Not a total deal breaker, but not really much better than my current si…

One option could be to have Python installed in a separate location specifically for this purpose and to NOT include it in PATH. Then it is "out-of-sight" of brew and such packages and sort. You can even make the entire location read-only once you are done with installation of Python + pipx etc.

That would definitely work.

Re: How to improve Python packaging

#102

Post by earthling8118 »

oconnor663 wrote: post #34
nixpulvis wrote: post #23

Earlier quoted context omitted.

Ruby gems and Cargo are pretty awesome. Also, I find I like Arch linux's Pacman quite a bit, though that's a slightly different use case where versioning isn't resolved.

Cargo is a gold standard imho, but there are definitely some simplifying decisions it makes that might not fit Python: - Dependencies and executables can run arbitrary code during build. For example, Cargo knows almost nothing about how to build C code, and the common workflow is to pull in the popular `cc` library for this and call it in your build.rs. - There's mostly no such thing as "installing a library", and ea…

I don't think the final build output being statically linked has any connection to a virtualenv. I've only ever used Python's virtualenvs to avoid having to install dependencies globally. When using Cargo you don't need to worry about that because their data is saved per-project already and Cargo can sort itself out when you invole a command

Re: How to improve Python packaging

#104

Post by di4na »

neillyons wrote: post #83
Lyngbakr wrote: post #21

It seems most people agree that Python's packaging isn't great, but, conversely, is there a language where most people agree that the approach to packaging is awesome? I mean, what's the gold standard to aspire to?

I'm a big fan of `mix` for Elixir. https://hexdocs.pm/mix/Mix.html

This. Mix is basically a mix (pun unintended) of Leiningen and Bundler. It works. Quite well.

Part of this is due to erlang solving a lot of the problems upfront (releases) and then the Hex team handling a lot of the rest.

But in general, the "Yehuda Katz" lineage (Bundler, Cargo, Yarn v1, mix as José used to be mentored by Yehuda) have pretty good tools worth copying. At least as a base.

Re: How to improve Python packaging

#106

Post by fbdab103 »

qbasic_forever wrote: post #15
woodruffw wrote: post #11

It's not very well documented, but the PyPA tools do provide a unified experience when used correctly. Here's a PyPA project (FD: one I work on) that uses a single pyproject.toml to handle all aspects of packaging (and most non-packaging tool configuration, to boot)[1]. With a single file like that, the only thing you need to do to start a local development environment is: python -m venv env && . env/bin/activate pyt…

Nowhere in pypa documentation is your simple workflow described or mentioned. Instead it's a jumble of links to a myriad of tools including hatchling, flit, pdm, etc. and basically just a shoulder shrug, 'I don't know, figure it all out yourself' message. This article makes a great point that the current pypa 'guidance' is too confusing and vague for actual end users (i.e. people that don't work directly on pypa or h…

A few weeks ago, I was attempting to help introduce someone to Python. I felt embarrassed at trying to paper-over how complicated and confusing the initial configuration is for a newbie. No officially endorsed best-practices to which I could point. Half a dozen tribes with their own competing solutions and guidance.

A definitive statement from Python.org as to The Way would go so far.

Re: How to improve Python packaging

#107

Post by fbdab103 »

woodruffw wrote: post #55
Izkata wrote: post #54

Earlier quoted context omitted.

By default (at least in python3 nowadays) it also excludes pip, setuptools, distribute, and wheel; you need "--all" to include them in the output.

Oh, that's news to me! I stand corrected. (The point about other development tooling is, I believe, still accurate -- if you e.g. have `black` installed, `pip freeze` will show it.)

Which is a huge limitation of many of the other tools. I have some beef with poetry, but it did at least get one thing correct: differentiating between the code required libraries and the development tooling (pytest, black, etc). There are hacky workarounds to accomplish this with other tools, but codifying this differentiation is incredibly valuable.

Re: How to improve Python packaging

#108

Post by jessekv »

groestl wrote: post #95
matsemann wrote: post #7

Because of the insanity of python I run everything in Docker (through compose). No more issues with it not working on some dev's computer because of a missing wheel, or that they need to have X and Y c++ toolchain packages installed locally etc. No more trying to fix a broken setup after upgrading python or poetry versions, just "docker compose build" and you're up and running. No spending days getting a freshly clon…

I secretly believe Docker only exists because of how f'up Python's distribution story is.

I also have often wondered how much Docker owes its success to the shortcomings of python.

Re: How to improve Python packaging

#109

Post by gen220 »

Lyngbakr wrote: post #21

It seems most people agree that Python's packaging isn't great, but, conversely, is there a language where most people agree that the approach to packaging is awesome? I mean, what's the gold standard to aspire to?

I think Go (if you started using it in the `go mod` era) and Rust have the best stories around package management.

The interface around `go mod` is kind of confusing, but I have actual trust in the dependency graph it generates for me. Cargo has, afaict, nailed both the interface and trust in what's going on under-the-hood.

In the Python world, Poetry isn't too bad. It's terribly slow in comparison to `go mod` or Cargo, but I generally trust and understand what's happening on the inside, and it's interface is fairly legible to newcomers.

Re: How to improve Python packaging

#110

Post by agumonkey »

baq wrote: post #87
agumonkey wrote: post #72

Earlier quoted context omitted.

where is the install .[dev] format/syntax defined ? I was trying to find what was possible and how to make sense of it in setup.cfg tools I found one mention in the docs but no more.

you may have done pip install jupyter[notebook] this is the same thing, just for . (current directory) and dev variant.

yeah but I'm missing data on how variant interacts with various requirements defined in setup.cfg (testing dependencies for instance)
Post reply on HN