Live data from Hacker News

PyPI Blog: Releases now reject new files after 14 days

blog.pypi.org

31–40 of 61 posts

Re: PyPI Blog: Releases now reject new files after 14 days

#31
post #28

This seems like common sense configuration management 101. If I download v1.2 and it’s been published then it should be considered released and not modifiable. With exceptions for ‘dev’ releases of course. I have never published anything on PyPI but I would expect there is a publish button and finalize (?) optional button that if not checked after 14 days makes it final ?

Are there any package managers that have that kind of publish/finalize flow? Every one I’m aware of works either as a one-shot (you have to submit everything in one push) or lets you keep adding new assets forever (other, obviously, than PyPI with the addition of this 14 day wall).

Doesn’t that sound bizarre? I have never heard of such a thing. The builds should be immutable

Re: PyPI Blog: Releases now reject new files after 14 days

#32
post #17

14 days is still too long if you ask me. Releases should be immutable.

Published files within a release are immutable. The time limit is needed because a release can contain different binary wheels for different architectures. Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners. Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their gen…

[deleted]

Re: PyPI Blog: Releases now reject new files after 14 days

#33

Earlier quoted context omitted.

I feel you’re quibbling over semantics here. In concept why can’t the full set of files in a release be a single, one-way hash value, with both adding or releasing changing the hash value?

The semantics are important. A release is composed of multiple distributions (sets of files). Each distribution does have a one-way hash value. This is what you lock to. Adding files does change the value of that hash. Files cannot be removed or changed. You lock to the distribution instead of the release so your build doesn't have to download distributions for platforms you're not using.

Which is the entire problem. "Release" should be a signed hash of a manifest file that contains hashes of all the other files (in case of Git that's what commit does).

PyPi just decided to do what VCSes already did, worse

Re: PyPI Blog: Releases now reject new files after 14 days

#34

14 days still seems way too long to me. As a user I thought releases on pypi were immutable!

Files are immutable on PyPI, releases are not (because releases are comprised of a set of files, and files are uploaded one by one). This is unintuitive, but the TL;DR is that files will never change on PyPI, but (previously) a user could upload a new file to a release years after their last upload to that release. This has some legitimate use cases (like allowing people to support new Python versions without bumping…

> (like allowing people to support new Python versions without bumping a package’s version)

...why Python is just breaking compatibility so bad with new version it needs that ?

Re: PyPI Blog: Releases now reject new files after 14 days

#35
post #17

Earlier quoted context omitted.

Published files within a release are immutable. The time limit is needed because a release can contain different binary wheels for different architectures. Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners. Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their gen…

Seems like better setup would be you stage a release and upload but once it’s promoted, its immutable.

I agree. I am not sure why they dont "just" wait until all the releases are ready before publishing and give a 14 day "deadline to publish" instead of a potential security vulnerability.

I understand wanting to get things out ASAP. but two weeks is not the end of the world to wait as a consumer of professional software packages.

Re: PyPI Blog: Releases now reject new files after 14 days

#36
Python packaging, the most convoluted way of creating simple zip files imaginable.

The tool fragmentation is insane, the demand to create "source distributions" was maybe funny in 2002 but just a hindrance now.

Packages no longer build since distutils was ripped out and upstream replaced it with meson etc.

Since building from source no longer works, which is profitable for third party vendors like Conda, "wheels" are uploaded. And they cannot be built on the server since the whole "scientific" ecosystem is perpetually broken. And they are separate artifacts, leading to the above problem.

Shipping checksummed tar archives is of course it not possible, that would hurt the income streams of the package profiteers.

Re: PyPI Blog: Releases now reject new files after 14 days

#37

Earlier quoted context omitted.

Hash pinning (already) works, and this change is all about when you as a PyPI user do not use hash pinning for installing releases, when you pin just release version for example. The release consists of one sdist and zero or more wheels. Until now you were able to upload additional wheels at later time.

Very pedantic of me, but I figure it’s interesting to note: technically a release on PyPI can have zero files or even one or more wheels but no sdist. The former is a degenerate case that users don’t normally see, and the latter happens if the user chooses to only upload wheels (or their sdist upload fails for whatever reason). (This doesn’t change your observations at all! Just as a demonstration of how Python packa…

I'd argue that the latter's a degenerate case as well--for the somewhat nitpicky/minor reason that package managers' errors are extremely poor at indicating the difference between "the version you asked for doesn't exist" and "there are no artifacts compatible with your platform at the version you asked for (and, for bonus points, here are some artifacts that are compatible with your platform)".

Something that I wish was included in PEP-694 is the notion of a "tombstone" invalid/empty sdist artifact to deal with that case, to make it easier for package managers to interpret and surface those different failure classes.

Like, sure, this is a package manager UX problem, but it's been happening for long enough that I feel like it might be time for PyPI to help managers do better.

Re: PyPI Blog: Releases now reject new files after 14 days

#38
It's kind of hilarious how everything that has to do with Python is so obviously wrong, with the obviously right way of doing things being right there, on the service, having been around since before Python even existed... and yet, Python will fail spectacularly every time.

If you ever used Maven, NPM or... I can't think about any other tool that doesn't automatically check checksums and signatures. Any Linux package manager ever used... Python's Wheel format has provisions for checksums and signatures! But they aren't checked.

Instead Python gets absurdly ineffective workarounds that will probably inconvenience a few developers and will do zilch for users.

Re: PyPI Blog: Releases now reject new files after 14 days

#39

Earlier quoted context omitted.

I feel you’re quibbling over semantics here. In concept why can’t the full set of files in a release be a single, one-way hash value, with both adding or releasing changing the hash value?

Because uploads to PyPI are not atomic. They’re now capped within a 14 day window, but it would be extremely confusing to users to have the “release” hash of their dependencies change repeatedly. (Also: we’d need to determine what it even means to hash a set of files. Do we order by canonicalized filename, by upload time, etc. Each of these has surprising implications!) Edit: to be clear, it’s not ideal or fully inte…

These things you’re posing as design questions are already solved 1 million times over. If you don’t want to design something new, just use Git semantics. These are not excuses for inaction. Python package management has been a horrifying embarrassment for 15 years and there’s no excuse for the amount it’s been neglected as every other ecosystem has shown better ways.

If it weren’t for AI, Python would have died as a language ecosystem due to their perpetual neglect of this area. The python community clearly had no ability to solve this problem themselves. I got sick and left and started using better tools.

It’s only now that they’re the cornerstone of the world economy that other people are coming in and cleaning up their messes for them. It’s pitiful. Python failed and needed to be saved from itself.

Re: PyPI Blog: Releases now reject new files after 14 days

#40
post #17

Earlier quoted context omitted.

Published files within a release are immutable. The time limit is needed because a release can contain different binary wheels for different architectures. Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners. Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their gen…

Seems like better setup would be you stage a release and upload but once it’s promoted, its immutable.

There's no reason a public index should allow any "staging" time for releases. It's the whole point of the release: once it's done, it's sealed. Oh, you messed up? -- Either make a new release, or release patches for the old one. Do not use production release server as your testing ground. It's not meant to do that, it makes it unnecessarily complicated. Test whatever you release on your own, outside of production grounds.
Post reply on HN