Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

discourse.nixos.org

Post reply on HN
Print view View original

1–10 of 232 posts

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#4

Post by xyzzy_plugh »

avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

There's a lot of problems with reproducible builds. Filesystem paths, timestamps, deterministic build order to say the least. This is a pretty great achievement and I'm looking forward to a non-minimal stable ISO.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#5

Post by twisrkrr »

avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

The code has to be changed so that things like system specific paths, time of compilation, hardware, etc. Don’t cause the compiled program to be unique to that computer (meaning compiling the same code on a different computer will give you a file that still works but has a different md5 hash)

By being able to reproduce the file completely, down to identical md5 hashes, you know you have the same file the creator has, and know with certainty that the file has not been tampered with

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#7

Post by danbst »

avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

Just recently, there were large non-reproducible projects: python, gcc. Not sure where is the history of non-r13y.

---

There is Debian initiative to create bit-to-bit reproducible builds for all their software (well, all critical).

https://reproducible-builds.org/

R13y is akin to "computer proofs" in math -- if you don't have it, that's fine, but if you have it, that's awesome.

There are practical reasons to favor reproducibility too, but those are more for distro maintainers.

The fact that NixOS (not Debian) got this 100% is mostly because

- minimal image has a small subset of packages (https://hydra.nixos.org/build/146009592#tabs-build-deps)

- Nix tooling was created 15 years ago *exactly* for this, Nix is mad to make packages bit-to-bit rebuildable from scratch.

- Nix/Nixpkgs is growing in number of maintainers and got more funds

- Nix has fewer Docker/Snap pragmatics

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#8

Post by mananaysiempre »

avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

The GCC developers in particular were hostile to such efforts for a long time, IIRC. (This is a non-trivial issue because randomized data structures exist and can be a good idea to use: treaps, universal hashes, etc. I’d guess it also pays for compiler heuristics to be randomized sometimes. Incremental compilation is much harder to achieve when you require bit-for-bit identical output. Even just stripping your compile paths from debug info is not entirely straightforward.)

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#10

Post by bombcar »

xyzzy_plugh wrote: post #4
avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

There's a lot of problems with reproducible builds. Filesystem paths, timestamps, deterministic build order to say the least. This is a pretty great achievement and I'm looking forward to a non-minimal stable ISO.

Yeah even the “gcc compiled Jan 23, 2021 at 11:23AM” messages you often see breaks deterministic builds.
Post reply on HN