Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Git archive checksums may change

github.blog

Post reply on HN
Print view View original

211–220 of 255 posts

Re: Git archive checksums may change

#211

Post by KyeRussell »

blueflow wrote: post #45
vlovich123 wrote: post #17

Hyrum's Law strikes again. It kind of doesn't matter what you document. If you weren't randomizing your checksum previously [1], you can't just spring this on the community and blame it for the fallout. I'm more shocked that there's resistance from the GitHub team saying "but we documented this isn't stable". Default stance for the team should be rollback & reevaluate an alternate path forward when the scope is this…

But look at it from the other side. Users that don't read your documentation and expect your software to work like they imagined are just a huge pain in the ass.

Do you work for Google?

Re: Git archive checksums may change

#212

Post by c4mpute »

shakow wrote: post #206
viraptor wrote: post #175

Earlier quoted context omitted.

The certificate guarantees the source of the file, not the trust you should put in its contents. I can upload malware as a github project release file and https doesn't change that you shouldn't download/run it. For software distribution this actually sometimes goes the other way - debian/ubuntu uses http (no s) for their packages, because the content itself is signed by the distribution and this way you can easily c…

> I can upload malware as a github project release file and https doesn't change that you shouldn't download/run it. If you can't trust the archive published by the owner themselves, you are already screwed; a stable hash will just make sure that you trust harder that you are, indeed, downloading contaminated code. I'm not sure most people here understand how checksums/hashs work, what they protect you against, and w…

Software published via GitHub isn't really "published by the owner". The owner typically doesn't control what GitHub does and doesn't always control his own GitHub account.

It isn't only that people don't know what checksums, hashes, and signatures do, it is also problematic that they blindly trust or ignore middlemen. Most supply chain "security" is security theater, almost never is something vetted end-to-end.

Re: Git archive checksums may change

#213

Post by philipwhiuk »

naikrovek wrote: post #95
philipwhiuk wrote: post #76

Yet another reason why GitHub is not a good Artifactory/Nexus replacement. Anyone remember the crazyness when Homebrew had problems with using GitHub for the same thing?

this is a git behavior, not a GitHub behavior. files uploaded to GH Packages are not modified by GitHub. only the "Source Code (.zip)" and "Source Code (.tgz)" files that are part of releases and tags are affected because git generates them on demand, and git does not guarantee hash stability. if you upload a package to GH Packages or upload a release asset to a GitHub releases those are never modified, and you can r…

No, it's not.

GitHub chooses to do this. It's GitHub's choice to generate Source Code files on demand rather than when the release is made. It's a way of reducing their disk usage at the cost of this kind of potential problem.

The problem is they also presented it as if it was a stable reference. If people knew it was not stable they would have done what the Bazel devs are now talking about doing, which is also uploading the source code at release time, as an artifact (which is how it works on Nexus).

Re: Git archive checksums may change

#214

Post by philipwhiuk »

lucb1e wrote: post #71

I didn't even know I should be depending on compression, file ordering, created-at file metadata, etc. being stable when pressing 'download repository as zip' (if I understand correctly what this is about, since the article doesn't really say). Perhaps it could be stable due to caching for a while after you first press it, but when it gets re-generated? I'm very surprised this was reproducible to begin with, given ho…

There are lots of methods to solve this problem - I imagine this was just easiest at the time given it appeared to work. Bazel devs on the list are discussing the best approach going forward - a simple change is to upload a fixed copy as a release artifact.

Re: Git archive checksums may change

#215

Post by nilsbunger »

c4mpute wrote: post #91
nilsbunger wrote: post #72

Earlier quoted context omitted.

This is especially true of something like a git SHA, which is drilled into your head as THE stable hash of your code and git tree at a certain state. It should be expected that lots of tools use it as an identifier -- heck, I've done so myself to confirm which version of a piece of software is deployed on a particular machine, etc.

Yes, but not in this bug. I guess lots of people missed that distinction: The stable git SHA hash is the commit hash, which is an hash over gits internal representation of the commit object (containing a tree of all file hashes, and parents' hashes). The hash that pops out of 'git archive' has nothing whatsoever to do with the commit hash and was historically stable more or less by accident: git feeds all files to 't…

Oh interesting. But if an archive hash isn’t stable, how is it meant to be used? What’s it good for?

Re: Git archive checksums may change

#217

Post by rfoo »

VWWHFSfQ wrote: post #38
phphphphp wrote: post #32

Earlier quoted context omitted.

Consumers often mistake hasn’t changed for a commitment to never change: any sufficiently large product will be littered with these kind of implicit commitments made by the product to consumers that nobody has visibility into. You’re unfortunate that we were all relying on this commitment you’ve never made, but the quick reversion is the best we can hope for. People will theorise how this could have been avoided but…

At this point they'll be stuck on old git for all of eternity unless they just roll their own archive/compress step out of band so the old hashes still work. Yikes.

New git has a flag for keeping the old behavior, so it's not as bad.

Re: Git archive checksums may change

#218

Post by SuperSandro2000 »

jart wrote: post #102
thirtyseven wrote: post #79

Earlier quoted context omitted.

The setup instructions for almost [1] every [2] major [3] rule set [4] only provide one (GitHub) url in the Starlark blob you're supposed to copy and paste, so hard to blame users here. [1] https://github.com/bazelbuild/rules_jvm_external/releases/ta... [2] https://github.com/bazelbuild/rules_python/releases/tag/0.17... [3] https://github.com/bazelbuild/rules_java/releases/tag/5.4.0 [4] https://github.com/bazelbuild/…

I agree. The Bazel developers failed in their leadership.

From a distro maintainer perspective every project that is only buildable with bazel is an absolute nightmare.

Re: Git archive checksums may change

#219

Post by rfoo »

lucb1e wrote: post #71

I didn't even know I should be depending on compression, file ordering, created-at file metadata, etc. being stable when pressing 'download repository as zip' (if I understand correctly what this is about, since the article doesn't really say). Perhaps it could be stable due to caching for a while after you first press it, but when it gets re-generated? I'm very surprised this was reproducible to begin with, given ho…

> gpg seems to be what all of them use

GPG signs a hash of the message with the private key, and you verify that the signature matches the file hash.

Oh wait, what hash? :clown:

Re: Git archive checksums may change

#220

Post by FeepingCreature »

kelnos wrote: post #208

The thing I don't get is how this ever worked. The change was upstream from git itself, and it was to use the builtin (zlib-based) compression code in git, rather than shelling out to gzip. But would the gzip binary itself give reproducible results across versions of gzip (and zlib)? Intuition seems to suggest it wouldn't, at least not always. And if not, was the "strategy" just to never update gzip or zlib on GitHub…

gzip is 28 years old. I don't think the output changes anymore.
Post reply on HN