Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Git archive checksums may change

github.blog

Post reply on HN
Print view View original

71–80 of 255 posts

Re: Git archive checksums may change

#71

Post by lucb1e »

I didn't even know I should be depending on compression, file ordering, created-at file metadata, etc. being stable when pressing 'download repository as zip' (if I understand correctly what this is about, since the article doesn't really say). Perhaps it could be stable due to caching for a while after you first press it, but when it gets re-generated? I'm very surprised this was reproducible to begin with, given how much trouble other projects have with that.

For projects where I verify the download, gpg seems to be what all of them use (thinking of projects like etesync and restic here). Interesting that so many people relied on a zip being generated byte-for-byte identically every time.

Re: Git archive checksums may change

#72

Post by nilsbunger »

vlovich123 wrote: post #17

Hyrum's Law strikes again. It kind of doesn't matter what you document. If you weren't randomizing your checksum previously [1], you can't just spring this on the community and blame it for the fallout. I'm more shocked that there's resistance from the GitHub team saying "but we documented this isn't stable". Default stance for the team should be rollback & reevaluate an alternate path forward when the scope is this…

This is especially true of something like a git SHA, which is drilled into your head as THE stable hash of your code and git tree at a certain state. It should be expected that lots of tools use it as an identifier -- heck, I've done so myself to confirm which version of a piece of software is deployed on a particular machine, etc.

Re: Git archive checksums may change

#73

Post by capableweb »

daniealapt wrote: post #48

Any change breaks a workflow - https://xkcd.com/1172/

True, small percentage will always be impacted by even the tiniest of change. But this was not that, checksums all over the place started breaking, as lots of FOSS is hosted on GitHub and lots of infrastructure depends on checksums remaining the same, otherwise they error out (correctly).

Re: Git archive checksums may change

#74

Post by johnklos »

vtbassmatt wrote: post #27
vtbassmatt wrote: post #20

Hey folks. I'm the product manager for Git at GitHub. We're sorry for the breakage, we're reverting the change, and we'll communicate better about such changes in the future (including timelines). Also posted here: https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

We updated our Git version which made this change for the reasons explained. At the time we didn't foresee the impact. We're quickly rolling back the change now, as it's clear we need to look at this more closely to see if we can make the changes in a less disruptive way. Thanks for letting us know.

[flagged]

Re: Git archive checksums may change

#75

Post by groestl »

VWWHFSfQ wrote: post #38
phphphphp wrote: post #32

Earlier quoted context omitted.

Consumers often mistake hasn’t changed for a commitment to never change: any sufficiently large product will be littered with these kind of implicit commitments made by the product to consumers that nobody has visibility into. You’re unfortunate that we were all relying on this commitment you’ve never made, but the quick reversion is the best we can hope for. People will theorise how this could have been avoided but…

At this point they'll be stuck on old git for all of eternity unless they just roll their own archive/compress step out of band so the old hashes still work. Yikes.

They could also brownout the implied contract over a longer timespan.

Re: Git archive checksums may change

#77

Post by capableweb »

swarfield wrote: post #7

They have broken almost every open source project that builds external deps. Also broke homebrew apparently.

Good test that the tooling actually works when the checksums are incorrect :) If your "build from source" tool/workflow DIDN'T break, I'd be worried.

Re: Git archive checksums may change

#78

Post by Denvercoder9 »

swarfield wrote: post #10
nick__m wrote: post #9

Earlier quoted context omitted.

I prefer that tool be adapted to be more resilient and not depend on github particular implementation.

Using SHA hashes when building guarantees that the code that you are building is what you think it is. How else would you verify dependencies like this, GPG signatures would have the same issue if you change the underlying bits.

I wouldn't check the hash of the compressed archive, but of the actual files themselves. It's a bit more metadata, but it's also a lot more robust, and allows you to detect changes after unpacking as well.

Re: Git archive checksums may change

#79

Post by thirtyseven »

jart wrote: post #18
medellin wrote: post #12

Im thinking of all the bazel build rules that are about to break from my last company. Someone will have a fun day updating hundreds of hashes.

If they're using multiple URLs like a good Bazel user then they shouldn't be impacted.

The setup instructions for almost [1] every [2] major [3] rule set [4] only provide one (GitHub) url in the Starlark blob you're supposed to copy and paste, so hard to blame users here.

[1] https://github.com/bazelbuild/rules_jvm_external/releases/ta...

[2] https://github.com/bazelbuild/rules_python/releases/tag/0.17...

[3] https://github.com/bazelbuild/rules_java/releases/tag/5.4.0

[4] https://github.com/bazelbuild/rules_scala

Re: Git archive checksums may change

#80

Post by lozenge »

denom wrote: post #40
vtbassmatt wrote: post #20

Hey folks. I'm the product manager for Git at GitHub. We're sorry for the breakage, we're reverting the change, and we'll communicate better about such changes in the future (including timelines). Also posted here: https://github.com/bazel-contrib/SIG-rules-authors/issues/11...

In my particular use-case, I'm using a set of local dev tools hosted as a homebrew tap. The build looks up the github tar.gz release for each tag and commits the sha256sum of that file to the formula What's odd is that all the _historical_ tags have broken release shasums. Does this mean the entire set of zip/tar.gz archives has been rebuilt? That could be a problem, as perhaps you cannot easily back out of this chan…

They never really stored them, they were always generated by some code (maybe with a cache layer in front). The code changed in a way that changed the bytes in the tar.gz without affecting their contents-when-extracted.
Post reply on HN