Github devs cannot point to their git commit, because Github is not open source.
Git archive checksums may change
241–250 of 255 posts
Re: Git archive checksums may change
#242Earlier quoted context omitted.
gzip is 28 years old. I don't think the output changes anymore.
There is no reason to believe that it won't. Even after 28 years, there could be improvements merged for the compressor. Or perhaps especially after 28 years - we have a lot more memory now but it is slower when compared to our CPUs than it used to be so there is most likely room for tuning. Similar for patches that make use of newer CPU instructions - why would you expect them to take care to produce the exact same…
That's the whole point, it wasn't an enforced contract but just happened to not change in a long time so it was assumed to be part of the contract. The majority of users don't know how exactly GitHub is serving these archives, they just assume (incorrectly, but reasonably) if they download from this URL they'll always get the same archive bit for bit. That assumption has grown stronger and stronger over time the longer they remained the same, until today.
Re: Git archive checksums may change
#243Earlier quoted context omitted.
I agree. The Bazel developers failed in their leadership.
From a distro maintainer perspective every project that is only buildable with bazel is an absolute nightmare.
I created https://gist.github.com/jart/082b1078a065b79949508bbe1b7d8ef... to solve that, by turning bazel projects into makefiles. The problem is the bazel team has broken the apis that make it possible so many times since then because they reacted very negatively to the idea.
Re: Git archive checksums may change
#244Earlier quoted context omitted.
Nixpkgs' so-called binary cache actually also caches source tarballs. Any Nix users out there who ran updates during the change? Did cache hits save you? Did cache misses break your builds?
Nixpkgs’s fetchFromGitHub function hashes the contents of GitHub archives after unpacking, so it’s unaffected.
I should have remembered this! Nixpkgs committers are consistently mindful of things like this in code reviews.
Re: Git archive checksums may change
#245Earlier quoted context omitted.
You're not wrong, but you're also not being realistic. Nix is not the only system that takes this approach. The Go modules "directory hash" is roughly equivalent, although we defined it in terms of somewhat more standard tooling: it is the output of sha256sum $(find . -type f | sort) | sha256sum I am not here advocating that everyone switch to this basic directory hash either, because it's not a solution to the more…
When you say it is not the right end-to-end solution for all cases, I am wondering what case you have in mind that a NAR Hash would not be suitable for. If you adopt Nix fully, the .narinfo file that cache.nixos.org (a Nix substituted) serves that is signed, contains both the NAR Hash and the hash of the NAR Archive File as well. Additionally, NAR packs and unpacks deterministically, and you can read the implementati…
> If you adopt Nix fully, ...
The case where Nix is not adopted fully is the one I have in mind.
Re: Git archive checksums may change
#246Earlier quoted context omitted.
When you say it is not the right end-to-end solution for all cases, I am wondering what case you have in mind that a NAR Hash would not be suitable for. If you adopt Nix fully, the .narinfo file that cache.nixos.org (a Nix substituted) serves that is signed, contains both the NAR Hash and the hash of the NAR Archive File as well. Additionally, NAR packs and unpacks deterministically, and you can read the implementati…
> If you adopt Nix fully, ... The case where Nix is not adopted fully is the one I have in mind.
This is the only case then?
Re: Git archive checksums may change
#247Earlier quoted context omitted.
Alternatively, they could extract the compression code and maintain it for repo tags created before the git algo update release date. Isn’t that the only humane course given all that depends on this?
That's my thought as well. They could also potentially retroactively generate the source tarballs using the old method for every possible repository/tag on Github, store it, and serve that, and then only generate it on-demand for new tags, but I doubt they'll do that. They might though, given this is what led to the problem in the first place (ie; the on-demand generation vs generating on push+storing).
That seems wasteful. Many projects do not actively advertise the GitHub tag downloads, and instead have their own stored and stable tarballs (or other distributions). And I suppose many users of those auto-generated downloads don’t care about their checksums.
Re: Git archive checksums may change
#248Earlier quoted context omitted.
> If you adopt Nix fully, ... The case where Nix is not adopted fully is the one I have in mind.
This is the only case then?
My point is about (1) the broader ecosystem of tools that may need to interoperate and have easy access to "SHA256 the whole file" and (2) the fact that not everything is a tar file that the Nix tools can process. So yes, that's the "only" case.
Re: Git archive checksums may change
#249Earlier quoted context omitted.
This is the only case then?
My point is about (1) the broader ecosystem of tools that may need to interoperate and have easy access to "SHA256 the whole file" and (2) the fact that not everything is a tar file that the Nix tools can process. So yes, that's the "only" case.
So what about the IPFS CAR format? https://car.ipfs.io/, it would fulfill a lot of what I expect from NAR too. NAR or CAR, I don't care, I believe the content is what matters, not the container format.
If I have a box with an apple in it, I don't care about the box, I care about the apple inside. If it's not an apple, I don't want to eat it.
Re: Git archive checksums may change
#250Github support, please checkout: https://news.ycombinator.com/item?id=34606345