151–160 of 479 posts
Earlier quoted context omitted.
The entire reason people choose "permissive licenses" is so that it won't last forever. At best, the community can fork the old version without any future features. Only viral licenses are forever.
I think you are making a good point, but please don't use the old Steve Baller FUD term, "viral." Copyleft is a better term
The real thing that I hope someone is able to solve is downloading such huge amounts of unnecessary code. As I understand, the bulk of the torch binary is just a huge nvfatbin compiled for every SM under the sun when you usually just want it to run on whatever accelerators you have on hand. Even just making narrow builds of like `pytorch-sm120a` (with stuff like cuBLAS thin binaries paired with it too) as part of a h…
Another piece is that PyPI has no index— it's just a giant list of URLs [1] where any required metadata (eg, the OS, python version, etc) is encoded in the filename. That makes it trivial to throw behind a CDN since it's all super static, but it has some important limitations: - there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info) - there's no way to get hashes…
> there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info)
Not true for wheels; PyPI implements https://peps.python.org/pep-0658/ here. You can pre-download just the dependency info instead.
For sdists, this is impossible until we can drop support for a bunch of older packages that don't follow modern standards (which is to say, including the actual "built" metadata as a PKG-INFO file, and having that file include static data for at least name, version and dependencies). I'm told there are real-world projects out there for which this is currently impossible, because the dependencies... depend on things that can't be known without inspecting the end user's environment. At any rate, this isn't a PyPI problem.
> there's no way to get hashes of the archives
This is provided as a URL fragment on the URLs, as described in https://peps.python.org/pep-0503/. Per PEP 658, the hash for the corresponding metadata files is provided in the data-dist-info-metadata (and data-core-metadata) attributes of the links.
But yes, there is no reverse-search support.
Probably the more useful blog post: https://astral.sh/blog/introducing-pyx
Thanks that’s bit less cryptic than the linked page. Still don’t get how they are solving what they claim to solve.
Is this going to solve the combinatorial explosion of pre-building native dependencies for every possible target? Python should get rid of its training wheels :^) https://kristoff.it/blog/python-training-wheels/
(As the user in question: three times.)
> Leverage progress in the systems programming ecosystem to create repeatable builds. Turn prebuilt binaries from “sources” into cacheable artifacts that can be deleted and reconstructed at will. Institute a way of creating secondary caches that can start shouldering some of the workload.
This doesn't avoid the need for the wheels to exist and be publicly available. People running CI systems should figure out local caching that actually works, sure. But if you delete that cacheable artifact on the public PyPI website for something like, I don't know, numpy-2.3.2-cp312-cp312-win_arm64.whl, you're going to be re-creating it (and having it downloaded again) constantly. Windows users are just not going to be able to build that locally.
And you know, space usage isn't the problem here — we're talking about a few hard drives' worth of space. The number of downloads is the problem. Outside of CI, I guess that's mostly driven by end users defaulting to the latest version of everything, every time they make a new virtual environment, rather than using whatever's in their package installer's cache. I do know that uv makes the latter a lot easier.
All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
Hopefully at the point the community is centralized enough to move in one direction.
All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
I share your concern but I have saved so much time with uv already that I figure ill ride it till the VC enshitification kills the host. Hopefully at the point the community is centralized enough to move in one direction.
Earlier quoted context omitted.
The repository API allows server-driven content negotiation[1], so pyx can service specialized requests while also honoring the normal 503/691 ones. [1]: https://packaging.python.org/en/latest/specifications/simple...
Ah. > honoring the normal 503/691 ones. Embrace > pyx can service specialized requests Extend ... ;)
Earlier quoted context omitted.
That was 7 years ago. If it's not coming back, the CLI should make that clear, instead of giving a temporary "cannot connect" message that implies it could work, if you wait a minute and try again.
It was three years ago; 2018 is when they considered removing the command, not when the search API was actually removed from PyPI. And this is part of the interlocking considerations I mentioned: there are private indices that supply the XML-RPC API, and breaking them doesn't seem justifiable[1]. Edit: fixed the link. [1]: https://github.com/pypa/pip/issues/5216#issuecomment-1235329...
It's so wild to me that so many people are apparently against making a user-friendly update. The whole thing seems very against pep8 (its surprising, complicated, non-specific, etc)
Earlier quoted context omitted.
> pyx uses the same PEP 503/691 interfaces as every other Python index ... Then how can it make decisions about how to serve the package request that PyPI can't? Is there not some extension to the protocol so that uv can tell it more about the client system?
The repository API allows server-driven content negotiation[1], so pyx can service specialized requests while also honoring the normal 503/691 ones. [1]: https://packaging.python.org/en/latest/specifications/simple...
> Why is it so hard to install PyTorch, or CUDA, or libraries like FlashAttention or DeepSpeed that build against PyTorch and CUDA? This is so true! On Windows (and WSL) it is also exacerbated by some packages requiring the use of compilers bundled with outdated Visual Studio versions, some of which are only available by manually crafting download paths. I can't wait for a better dev experience.
Stuff like that led me fully away from Ruby (due to Rails), which is a shame, I see videos of people chugging along with Ruby and loving it, and it looks like a fun language, but when the only way I can get a dev environment setup for Rails is using DigitalOcean droplets, I've lost all interest. It would always fail at compiling something for Rails. I would have loved to partake in the Rails hype back in 2012, but ov…