Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

discourse.nixos.org

Post reply on HN
Print view View original

31–40 of 179 posts

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#31

Post by Aerbil313 »

mihalycsaba wrote: post #3

Sorry for being dense, but I thought one of the main reason for nixos's existence is reproducibilty. I thought they have these kinds of things solved already. I have only ~2 hours experience with Nixos, wanted to try hyprland, I thought it would be easier on Nixos since hyprland needs a bit of setup and maybe it's easier to use someone else's config on nixos, than on some other distro. Finding a config was hard too,…

Check out https://github.com/donovanglover/nix-config . Flake based config with hyprland and cool stuff.

> at that point I gave up.

NixOS is not for the weak or time constrained, currently. Hopefully it will be one day. Still if you push through, you reap the benefits.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#32

Post by ahmedfromtunis »

Stupid question as I never worked on something like this before: why isn't reproducibility the default behavior?

I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time?

I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#33

Post by bravetraveler »

ahmedfromtunis wrote: post #32

Stupid question as I never worked on something like this before: why isn't reproducibility the default behavior? I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time? I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.

I don't develop enough to give a particularly good answer, but one example I've heard of involves timestamps

Imagine the program uses the current date or time as a value. When compiled at different moments, the bits change.

Same applies to anything where the build environment or timing influences the output binary

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#34

Post by kaba0 »

ahmedfromtunis wrote: post #32

Stupid question as I never worked on something like this before: why isn't reproducibility the default behavior? I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time? I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.

Parallelism. There might be actions that are not order-independent, and the state of the CPU might result in slightly different binaries, but all are correct.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#35

Post by Smaug123 »

ahmedfromtunis wrote: post #32

Stupid question as I never worked on something like this before: why isn't reproducibility the default behavior? I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time? I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.

Loads of things. Obvious ones where the decision is explicitly taken to be non-reproducible include timestamps and authorship information. There are also other places where reproducibility is implicitly broken by default: e.g. many runtimes don't define the order of entries in a hashmap, and then the compiler iterates over a hashmap to build the binary.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#36

Post by rgoulter »

candiddevmike wrote: post #24
KennyFromIT wrote: post #16

I've lived in the Red Hat ecosystem for work recently. How does this compare to something like... Fedora Silverblue? Ansible? Fedora Silverblue + Ansible?

Nix is a declarative OS, where you describe what the OS should look like, instead of Ansible where you give the OS steps to follow. Silverblue and Nix are orthogonal aside from being Linux distributions--Silverblue is attempting to change how software is delivered using only containers on an immutable host. If you're interested in an Ansible alternative that uses Jsonnet and state tracking to somewhat mimic Nix, chec…

> Nix is a declarative OS

I think precision is important.

"Nix" refers to the package manager (and the language the package manager uses).

Whereas it's "NixOS" that's the OS which makes use of Nix to manage the system configuration.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#37

Post by msm_ »

Thrir94994i wrote: post #11
onedognight wrote: post #7

Rebuilding the minimal ISO from source is an impressive milestone on the journey to a system that builds from source reproducibly. Guix had an orthogonal but equally impressive milestone on the same journey recently[0], bootstrapping a full compiler toolchain from a single reproducible 357 byte binary without any other binary compiler blobs. These two features may one day soon be combined to reproducibly build a full…

357 bytes for bootstrap compiler binary is VERY impressive!

If I remember correctly, this tiny binary is used to (reproducibly) bootstrap the next binary, which bootstraps the next binary, until eventually GCC can be compiled (and compile other software).

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#38

Post by Smaug123 »

msm_ wrote: post #37
Thrir94994i wrote: post #11

Earlier quoted context omitted.

357 bytes for bootstrap compiler binary is VERY impressive!

If I remember correctly, this tiny binary is used to (reproducibly) bootstrap the next binary, which bootstraps the next binary, until eventually GCC can be compiled (and compile other software).

The bootstrap chain is https://github.com/oriansj/stage0-posix-x86/blob/e86bf7d304b...

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#39

Post by mauricioc »

Reventlov wrote: post #4

For those wondering : it should be remembered that the reproducibility of Nix / NixOS / Nixpkgs is only a reproducibility of the sources: if the sources change, one is warned, but it is not a question of the reproducibility of the binaries (which can change at each build). This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically. Guix, Archlinux, Debian do the bina…

To emphasize chpatrick's point below, there are two definitions of "reproducibility" in this context:

* Input reproducibility, meaning "perfect cache invalidation for inputs". Nix and Guix do this perfectly by design (which sometimes leads to too many rebuilds). This is not on the radar for Debian and Arch Linux, which handle the rebuild problem ("which packages should I rebuild if a particular source file is updated?") on an ad-hoc basis by triggering manual rebuilds.

* Output reproducibility, meaning "the build process is deterministic and will always produce the same binary". This is the topic of the OP. Nix builds packages in a sandbox, which helps but is not a silver bullet. Nix is in the same boat as Debian and Arch Linux here; indeed, distros frequently upstream patches to increase reproducibility and benefit all the other distros. In this context, https://reproducible.nixos.org is the analogue of the other links you posted, and I agree Nix reports aren't as detailed (which does not mean binary reproducibility is worse on Nix).

Your comment can be misinterpreted as saying "Nix does not do binary reproducibility very well, just input reproducibility", which is false. That's the whole point of the milestone being celebrated here!

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#40

Post by dataflow »

onedognight wrote: post #7

Rebuilding the minimal ISO from source is an impressive milestone on the journey to a system that builds from source reproducibly. Guix had an orthogonal but equally impressive milestone on the same journey recently[0], bootstrapping a full compiler toolchain from a single reproducible 357 byte binary without any other binary compiler blobs. These two features may one day soon be combined to reproducibly build a full…

How long does a fully bootstrapped build take?
Post reply on HN