Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Git archive checksums may change

github.blog

Post reply on HN
Print view View original

131–140 of 255 posts

Re: Git archive checksums may change

#131

Post by bentley »

rsc wrote: post #123
vtbassmatt wrote: post #20

Hey folks. I'm the product manager for Git at GitHub. We're sorry for the breakage, we're reverting the change, and we'll communicate better about such changes in the future (including timelines). Also posted here: https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

Thanks for the quick rollback. I want to encourage you to think about locking in the current archive details, at least for archives that have already been served. Verifying that downloaded archives have the expected checksum is a critical best practice for software supply chain security. Training people to ignore checksum changes is training them to ignore attacks. GitHub is a strong leader in other parts of supply c…

I would also appreciate stronger advertising of the ability to turn a Git tag into a GitHub release and upload stable source code files to it. Maybe even a button in the GitHub releases interface to “generate source tarball and attach as stable tarball to this release.”

Re: Git archive checksums may change

#132

Post by metrognome »

I wonder if this incident will encourage our industry to build more robust forms of artifact integrity verification, or if we will instead codify the status quo of "we guarantee repos to be archived deterministically." To me, the latter seems like a more troubling precedent.

Re: Git archive checksums may change

#133

Post by chlorion »

VWWHFSfQ wrote: post #82
johnklos wrote: post #74

Earlier quoted context omitted.

[flagged]

Ironic that "open source packaging systems" rely on proprietary Microsoft hosting and distribution to function. I think you meant _poorly implemented_ open source packaging systems.

I am not sure how you could not rely on GitHub when packaging code that is hosted on GitHub.

My personal Gentoo ebuilds for example contain a URI variable that points to the GitHub auto-generated archives for projects that use GitHub for hosting. What am I supposed to do in this case?

The only option here is to setup a mirror and have a backup of the data. Packages that are in the official Gentoo repository do get mirrored, but overlays do not, and most people probably don't have the ability, time or money to setup their own mirroring service.

I agree that relying on GitHub sucks to be clear, but I don't think we can blame package managers for fetching code from it when people are hosting their projects there!

Re: Git archive checksums may change

#134

Post by mardifoufs »

catiopatio wrote: post #116
groestl wrote: post #66

Earlier quoted context omitted.

> every open source project that builds external deps and relies on checksumming ephemeral artefacts for integrity.

Source archives have never , in the entire history of open source, been considered ephemeral. GitHub unilaterally made that decision for their own convenience, and violated a decades-long universal community norm in the process.

I think this change only affects automatically (and dynamically) generated source archives, not those that are actually pushed to Github Releases beforehand.

Re: Git archive checksums may change

#135

Post by Denvercoder9 »

catiopatio wrote: post #116
groestl wrote: post #66

Earlier quoted context omitted.

> every open source project that builds external deps and relies on checksumming ephemeral artefacts for integrity.

Source archives have never , in the entire history of open source, been considered ephemeral. GitHub unilaterally made that decision for their own convenience, and violated a decades-long universal community norm in the process.

You could also say that some maintainers made that decision for their convenience of not having to build and upload source archives. It is possible to upload your own artifacts to a release on GitHub, and lots of projects do. Those are correctly treated as immutable by GitHub.

Re: Git archive checksums may change

#136

Post by bentley »

metrognome wrote: post #132

I wonder if this incident will encourage our industry to build more robust forms of artifact integrity verification, or if we will instead codify the status quo of "we guarantee repos to be archived deterministically." To me, the latter seems like a more troubling precedent.

We’ve regressed from the previous norm of open source projects providing stable source tarballs with fixed checksums, sometimes even with cryptographic signatures.

Re: Git archive checksums may change

#137

Post by clhodapp »

pxc wrote: post #88
frankjr wrote: post #5

GitHub will need to revert this change. They've just crippled pretty much every "from source" package manager out there.

Nixpkgs' so-called binary cache actually also caches source tarballs. Any Nix users out there who ran updates during the change? Did cache hits save you? Did cache misses break your builds?

I could be wrong but believe that nix should be safe for the most part because it does a recursive hash of the stuff it cares about on the extraction of these archives.

Re: Git archive checksums may change

#139

Post by Denvercoder9 »

pxc wrote: post #117
VWWHFSfQ wrote: post #82

Earlier quoted context omitted.

Ironic that "open source packaging systems" rely on proprietary Microsoft hosting and distribution to function. I think you meant _poorly implemented_ open source packaging systems.

> I think you meant _poorly implemented_ open source packaging systems. or under-resourced ones. If the upstream source only appears on GitHub, without formal release tarballs, your only options as a downstream packager are literally to get the source from GitHub or host your own mirror of every source tarball you build yourself.

Or get the source code using Git, which actually (by design) guarantees that its checksums are stable.

Re: Git archive checksums may change

#140

Post by shakow »

catiopatio wrote: post #122
shakow wrote: post #111

Earlier quoted context omitted.

By checking the hash of the extracted files. The hash of the archive is dependent on the order in which the file were compressed, the compression, some metadata, etc.

That’s expensive, complicated, exposes a greater attack surface, and requires new tooling to maintain considerably more complex metadata covering the full contents of source archives. For the entire multi-decade history of open source, the norm has been — for very good reason — that source archives are immutable and will not change. The solution here isn’t to change the entire open source ecosystem.

> That’s expensive, complicated,

That sounds like prejudice. Just as a test, I cloned the git repo, which took 29 seconds, then took its hash with `guix hash`, which took 0.387ms.

I think that if you can't handle a 0.4s delay in a build, you have problem problems.

Post reply on HN