Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Git archive checksums may change

github.blog

Post reply on HN
Print view View original

31–40 of 255 posts

Re: Git archive checksums may change

#31

Post by medellin »

ErikCorry wrote: post #28
medellin wrote: post #12

Im thinking of all the bazel build rules that are about to break from my last company. Someone will have a fun day updating hundreds of hashes.

Do they let Github generate the archives as one of the build rules instead of performing the archival and compression locally and uploading the result?

Correct. Silly stuff like this happens when you don’t have systems in place that make it easy to store your own artifacts. Additionally a lot of people just want to get things done as quick as possible even if you have the tools in place.

Re: Git archive checksums may change

#32

Post by phphphphp »

vtbassmatt wrote: post #27
vtbassmatt wrote: post #20

Hey folks. I'm the product manager for Git at GitHub. We're sorry for the breakage, we're reverting the change, and we'll communicate better about such changes in the future (including timelines). Also posted here: https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

We updated our Git version which made this change for the reasons explained. At the time we didn't foresee the impact. We're quickly rolling back the change now, as it's clear we need to look at this more closely to see if we can make the changes in a less disruptive way. Thanks for letting us know.

Consumers often mistake hasn’t changed for a commitment to never change: any sufficiently large product will be littered with these kind of implicit commitments made by the product to consumers that nobody has visibility into. You’re unfortunate that we were all relying on this commitment you’ve never made, but the quick reversion is the best we can hope for. People will theorise how this could have been avoided but c’est la vie — easy mistake that you’ve responded well to.

Re: Git archive checksums may change

#33

Post by jart »

medellin wrote: post #30
jart wrote: post #18

Earlier quoted context omitted.

If they're using multiple URLs like a good Bazel user then they shouldn't be impacted.

They did where applicable but i know that not all of them had multiple

Well now they know why it's so important. https://github.com/bazelbuild/bazel/commit/ed7ced0018dc5c5eb...

Re: Git archive checksums may change

#34

Post by ErikCorry »

swarfield wrote: post #10
nick__m wrote: post #9

Earlier quoted context omitted.

I prefer that tool be adapted to be more resilient and not depend on github particular implementation.

Using SHA hashes when building guarantees that the code that you are building is what you think it is. How else would you verify dependencies like this, GPG signatures would have the same issue if you change the underlying bits.

This seems like a weak argument.

Firstly SHA is not a secure hash.

Secondly if your build step involves uploading data to a third party then allowing them to transform it as they see fit and then checksumming the result then it's not really a reproducible build. For all you know, Github inserts a virus during the compression of the archive.

What am I missing?

Re: Git archive checksums may change

#35

Post by elesiuta »

doubleunplussed wrote: post #4

Ah, this will presumably break some Arch Linux AUR packages. Preparing for bug reports.

I always anticipated something like this could happen and it bothered me enough to create my own workflow [1] to archive, hash, and attach it to each release automatically for my AUR package. I can see how most people wouldn't notice/bother with such a small detail though, so I am not at all surprised by the fallout this caused.

[1] https://github.com/elesiuta/picosnitch/blob/master/.github/w...

Re: Git archive checksums may change

#37

Post by ArchOversight »

vlovich123 wrote: post #22
ArchOversight wrote: post #16

Earlier quoted context omitted.

a git checkout of the code at that particular tag hasn't changed. Just the tarball that git archive generates has.

The two main problems are: A) How do you catch tarballs that have extra files injected that aren't part of your manifest B) What does the performance of this look like? Certainly for traditional HDDs this is going to kill performance, but even for SSDs I think verifying a bunch of small files is going to be less efficient than verifying the tarball.

A wouldn't be an issue since you are checking out a git tag.

B would just be a normal git checkout, which already validates that all the objects are reachable and git tags (and commits for that matter) can be signed, and since the sha1 hash is signed as well it validates that the entire tree of commits has not been tampered with. So as long you trust git to not lie about what it is writing to disk, you have a valid checkout of that tag.

And if you do expect it to lie, why do you expect tar to not lie about what it is unpacking?

Re: Git archive checksums may change

#38

Post by VWWHFSfQ »

phphphphp wrote: post #32
vtbassmatt wrote: post #27

Earlier quoted context omitted.

We updated our Git version which made this change for the reasons explained. At the time we didn't foresee the impact. We're quickly rolling back the change now, as it's clear we need to look at this more closely to see if we can make the changes in a less disruptive way. Thanks for letting us know.

Consumers often mistake hasn’t changed for a commitment to never change: any sufficiently large product will be littered with these kind of implicit commitments made by the product to consumers that nobody has visibility into. You’re unfortunate that we were all relying on this commitment you’ve never made, but the quick reversion is the best we can hope for. People will theorise how this could have been avoided but…

At this point they'll be stuck on old git for all of eternity unless they just roll their own archive/compress step out of band so the old hashes still work. Yikes.

Re: Git archive checksums may change

#39

Post by forgotpwd16 »

metrognome wrote: post #13
frankjr wrote: post #5

GitHub will need to revert this change. They've just crippled pretty much every "from source" package manager out there.

Per the post, this was a change to git itself: https://github.com/git/git/commit/4f4be00d302bc52d0d9d5a3d47...

What was the thought behind this change?

Re: Git archive checksums may change

#40

Post by denom »

vtbassmatt wrote: post #20

Hey folks. I'm the product manager for Git at GitHub. We're sorry for the breakage, we're reverting the change, and we'll communicate better about such changes in the future (including timelines). Also posted here: https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

In my particular use-case, I'm using a set of local dev tools hosted as a homebrew tap.

The build looks up the github tar.gz release for each tag and commits the sha256sum of that file to the formula

What's odd is that all the _historical_ tags have broken release shasums. Does this mean the entire set of zip/tar.gz archives has been rebuilt? That could be a problem, as perhaps you cannot easily back out of this change...

Post reply on HN