[flagged]
PYX: The next step in Python packaging
91–100 of 479 posts
Re: PYX: The next step in Python packaging
#92Earlier quoted context omitted.
I agree. If any of the stuff was worthwhile to pursue, it would be merged into pip.
Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI. Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking a…
> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original).
FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
I'm not saying here that pip doesn't have some gnarly internal details, just that the bigger thing holding it back is the lack of maintainer resources.
> For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process.
I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.
That said, I do think there are other good examples to your point, like uv's ability to prefetch package metadata, I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.
Re: PYX: The next step in Python packaging
#93I do not trust Astral.
Much ad language.
They do not explain what an installation of their software does to my system.
They use the word "platform".
Re: PYX: The next step in Python packaging
#94I'm brushing up with Python for a new job, and boy what a ride. Not because of the language itself but the tooling around packages. I'm coming from Go and TS/JS and while these two ecosystems have their own pros and cons, at least they are more or less straightforward to get onboarded (there are 1 or 2 tools you need to know about). In Python there are dozens of tools/concepts related to packaging: pip, easy_install,…
You don't need to know most of those things. Until last year I used setup.py and pip exclusively for twenty years, with a venv for each job at work. Wheels are simply prebuilt .zips. That's about an hour of learning more or less.
Now we have pyproject.toml and uv to learn. This is another hour or so of learning, but well worth it.
Astral is stepping up because no one else did. Guido never cared about packaging and that's why it has been the wild west until now.
Re: PYX: The next step in Python packaging
#95Is there a big enough commercial market for private Python package registries to support an entire company and its staff? Looks like they're hiring for $250k engineers, starting a $26k/year OSS fund, etc. Expenses seem a bit high if this is their first project unless they plan on being acquired?
Its interesting because the value is definitely there. Every single python developer you meet (many of who are highly paid) has a story about wasting a bunch of time on these things. The question is how much of this value can Astral capture. I think based on the quality of their work, there's also an important component which is trust. I'd trust and pay for a product from them much more readily than an open source so…
Yeah they certainly generate a lot of value by providing excellent productivity tooling. The question is how they capture some of that value, which is notoriously hard with an OSS license. A non-OSS license creates the adobe trap on the other hand, where companies deploy more and more aggressive moetization strategies, making life worse and worse for users of the software.
Re: PYX: The next step in Python packaging
#96> Why is it so hard to install PyTorch, or CUDA, or libraries like FlashAttention or DeepSpeed that build against PyTorch and CUDA? This is so true! On Windows (and WSL) it is also exacerbated by some packages requiring the use of compilers bundled with outdated Visual Studio versions, some of which are only available by manually crafting download paths. I can't wait for a better dev experience.
Stuff like that led me fully away from Ruby (due to Rails), which is a shame, I see videos of people chugging along with Ruby and loving it, and it looks like a fun language, but when the only way I can get a dev environment setup for Rails is using DigitalOcean droplets, I've lost all interest. It would always fail at compiling something for Rails. I would have loved to partake in the Rails hype back in 2012, but ov…
I would recommend learning a little bit of C compilation and build systems. Ruby/Rails is about as polished as you could get for a very popular project. Maybe libyaml will be a problem once in a while if you're compiling Ruby from scratch, but otherwise this normally works without a hassle. And those skills will apply everywhere else. As long as we have C libraries, this is about as good as it gets, regardless of the language/runtime.
Re: PYX: The next step in Python packaging
#97Earlier quoted context omitted.
I might not be following: what would that protocol entail? pyx uses the same PEP 503/691 interfaces as every other Python index, but those interfaces would likely not be immediately useful to PyPI itself (since it already has them). > or is one simply paying for access to Astral's particular server that runs it? pyx is currently a service being offered by Astral. So it's not something you can currently self-host, if…
> pyx uses the same PEP 503/691 interfaces as every other Python index ... Then how can it make decisions about how to serve the package request that PyPI can't? Is there not some extension to the protocol so that uv can tell it more about the client system?
The repository API allows server-driven content negotiation[1], so pyx can service specialized requests while also honoring the normal 503/691 ones.
[1]: https://packaging.python.org/en/latest/specifications/simple...
Re: PYX: The next step in Python packaging
#98> Why is it so hard to install PyTorch, or CUDA, or libraries like FlashAttention or DeepSpeed that build against PyTorch and CUDA? This is so true! On Windows (and WSL) it is also exacerbated by some packages requiring the use of compilers bundled with outdated Visual Studio versions, some of which are only available by manually crafting download paths. I can't wait for a better dev experience.
This is the right direction for Python packaging, especially for GPU-heavy workflows. Two concrete things I'm excited about: 1) curated, compatibility-tested indices per accelerator (CUDA/ROCm/CPU) so teams stop bikeshedding over torch/cu* matrixes, and 2) making metadata queryable so clients can resolve up front and install in parallel. If pyx can reduce the 'pip trial-and-error' loop for ML by shipping narrower, hardware-targeted artifacts (e.g., SM/arch-specific builds) and predictable hashes, that alone saves hours per environment. Also +1 to keeping tools OSS and monetizing the hosted service—clear separation builds trust. Curious: will pyx expose dependency graph and reverse-dependency endpoints (e.g., "what breaks if X→Y?") and SBOM/signing attestation for supply-chain checks?
Re: PYX: The next step in Python packaging
#99I'm brushing up with Python for a new job, and boy what a ride. Not because of the language itself but the tooling around packages. I'm coming from Go and TS/JS and while these two ecosystems have their own pros and cons, at least they are more or less straightforward to get onboarded (there are 1 or 2 tools you need to know about). In Python there are dozens of tools/concepts related to packaging: pip, easy_install,…
I appreciate everything they’ve done but the group which maintains Pip and the package index is categorically incapable of shipping anything at a good velocity. It’s entirely volunteer based so I don’t blame them, but the reality is that it’s holding back the ecosystem. I suspect it’s also a misalignment of interests. No one there really invests in improving UX.
Indeed, they broke a few features in the last few years and made the excuse "we can't support them, we're volunteers." Well, how about stop breaking things that worked for a decade? That would take less effort.
They had time to force "--break-system-packages" on us though, something no one asked for.
Re: PYX: The next step in Python packaging
#100Astral is the coolest startup