Live data from Hacker News

PyPI Blog: Releases now reject new files after 14 days

blog.pypi.org

11–20 of 61 posts

Re: PyPI Blog: Releases now reject new files after 14 days

#11

There seems to be a severe lack of hash pinning in "modern" software ecosystems. We figured out how to do it 20+ years ago with Git bringing hash addressed storage to the masses. Coming from a different background it was very surprising for me to see things like docker images, packages and github actions being updated at the whim of upstream registry. I much prefer the philosophy where builds are fully offline and pr…

BitTorrent used hash pinning earlier than that even.

Re: PyPI Blog: Releases now reject new files after 14 days

#12

There seems to be a severe lack of hash pinning in "modern" software ecosystems. We figured out how to do it 20+ years ago with Git bringing hash addressed storage to the masses. Coming from a different background it was very surprising for me to see things like docker images, packages and github actions being updated at the whim of upstream registry. I much prefer the philosophy where builds are fully offline and pr…

Hash pinning (already) works, and this change is all about when you as a PyPI user do not use hash pinning for installing releases, when you pin just release version for example.

The release consists of one sdist and zero or more wheels. Until now you were able to upload additional wheels at later time.

Re: PyPI Blog: Releases now reject new files after 14 days

#13

There seems to be a severe lack of hash pinning in "modern" software ecosystems. We figured out how to do it 20+ years ago with Git bringing hash addressed storage to the masses. Coming from a different background it was very surprising for me to see things like docker images, packages and github actions being updated at the whim of upstream registry. I much prefer the philosophy where builds are fully offline and pr…

I’m not sure what this has to do with TFA: Python does have hash-pinning. TFA is not about modifying existing files on the index (PyPI doesn’t allow that), but about adding new files to a pre-existing release. But that doesn’t change the hash of older distributions on that release.

I feel you’re quibbling over semantics here.

In concept why can’t the full set of files in a release be a single, one-way hash value, with both adding or releasing changing the hash value?

Re: PyPI Blog: Releases now reject new files after 14 days

#16

Earlier quoted context omitted.

I’m not sure what this has to do with TFA: Python does have hash-pinning. TFA is not about modifying existing files on the index (PyPI doesn’t allow that), but about adding new files to a pre-existing release. But that doesn’t change the hash of older distributions on that release.

I feel you’re quibbling over semantics here. In concept why can’t the full set of files in a release be a single, one-way hash value, with both adding or releasing changing the hash value?

The semantics are important. A release is composed of multiple distributions (sets of files). Each distribution does have a one-way hash value. This is what you lock to. Adding files does change the value of that hash. Files cannot be removed or changed.

You lock to the distribution instead of the release so your build doesn't have to download distributions for platforms you're not using.

Re: PyPI Blog: Releases now reject new files after 14 days

#17

14 days is still too long if you ask me. Releases should be immutable.

Published files within a release are immutable.

The time limit is needed because a release can contain different binary wheels for different architectures.

Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners.

Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their generated files.

That window used to be unlimited, now it's 14 days.

That might seem like a long time, but it means more manual release processes still have time to coordinate, or release processes that need access to less common hardware that might require queuing for a while.

Re: PyPI Blog: Releases now reject new files after 14 days

#18
post #17

14 days is still too long if you ask me. Releases should be immutable.

Published files within a release are immutable. The time limit is needed because a release can contain different binary wheels for different architectures. Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners. Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their gen…

I'd be happier if it was 24 hours by default, with an increased timeout configurable behind 2FA.

Re: PyPI Blog: Releases now reject new files after 14 days

#19
post #17

14 days is still too long if you ask me. Releases should be immutable.

Published files within a release are immutable. The time limit is needed because a release can contain different binary wheels for different architectures. Consider the simplest case: your releases go out via GitHub Actions and separate wheels are built on the Windows, Linux, and macOS runners. Those won't all end at exactly the same time, so you need a release window during which they can finish and upload their gen…

Seems like better setup would be you stage a release and upload but once it’s promoted, its immutable.

Re: PyPI Blog: Releases now reject new files after 14 days

#20

Earlier quoted context omitted.

I’m not sure what this has to do with TFA: Python does have hash-pinning. TFA is not about modifying existing files on the index (PyPI doesn’t allow that), but about adding new files to a pre-existing release. But that doesn’t change the hash of older distributions on that release.

I feel you’re quibbling over semantics here. In concept why can’t the full set of files in a release be a single, one-way hash value, with both adding or releasing changing the hash value?

Because uploads to PyPI are not atomic. They’re now capped within a 14 day window, but it would be extremely confusing to users to have the “release” hash of their dependencies change repeatedly.

(Also: we’d need to determine what it even means to hash a set of files. Do we order by canonicalized filename, by upload time, etc. Each of these has surprising implications!)

Edit: to be clear, it’s not ideal or fully intentional that uploads to PyPI are not atomic. But it’s the status quo and any change will take years.

Post reply on HN