Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

That's true, I didn't think of that! D:

Well, you'd still check the signature, to make sure you have the same build that the debian repository has.

This is just an additional check that the debian repository has sane builds.

(If someone mucks around with the debian repositories but you aren't the target, you might or might not be under attack.)

It should be noted this was achieved by removing Python bytecode from the builds. This has a performance hit and should probably be reverted.

tl;dr: Debian's work is very important here, but NixOS' reproducibility aims are more general than Debian's and began more than 8 years earlier

Despite the fact that Debian (as a project) has shouldered far more of the work with upstream projects to make bit-identical reproducibility possible at build time, Debian (as a distro) doesn't have a design that makes this kind of reproducibility as feasible, practical, or robust at the level of a whole system or disk image in the way that NixOS has achieved here. To quote the Debian project itself[0]:

> Reproducible builds of Debian as a whole is still not a reality, though individual reproducible builds of packages are possible and being done. So while we are making very good progress, it is a stretch to say that Debian is reproducible.

Beyond the fact that some packages still have issues upstream and the basic technical problem of versioning (i.e., apt fetching binaries from online archives in a stateful way) Debian additionally struggles with an extremely heterogeneous and manual process of acquiring and uploading source packages[1]. Debian doesn't even have the resources to construct a disk image where the version of every package is pinned, short of archiving all the binaries (which is how they do ‘reproducible’ ISO production now[2]). But pulling down all of the pre-built binaries for your distro isn't really ‘reproduction’ in the same sense as ‘reproduction’ in Debian's (package-level) reproducibility project.

Some points of comparison

  • NixOS always fixes the whole dependency tree
  • Debian requires a ‘snapshot’ repository to fix a dependency tree
  • most NixOS packages are updated through automatic tools and all the build recipes are stored under version control in one place
  • Debian packages can be updated any way that suits their maintainers, and the build recipes/rules can be stored anywhere (it's the maintainer's job to keep them in version control if they want, then upload them to Debian repositories as source packages)
  • Nix (transparently!) caches both build outputs and package sources, which means
    ◦ if the original source tarballs (e.g., on GitHub or SourceForge) are unavailable, Nix won't even notice that if it can pull them from the ‘binary’ cache
    ◦ if there is no cache of the build outputs, Nix will automatically fall back to fetching and unpacking the sources from the upstream mirror
  • Debian's technical and community relationships to upstream source code are both less robust
    ◦ Debian requires manual management (creating and uploading) of complete source code archives in their own format[1]
    ◦ sometimes Debian infrastructure can't even reproduce upstream source code from their own archives[3]
    ◦ if Debian's source archives are unavailable for a package, there is just no way to build it (since source package archives also contain the build instructions, dependency metadata, etc.)

Actually reproducing a NixOS image is less manual and can be done without relying on any online Nix/NixOS-specific infrastructure, and this is a real advancement over what's possible with binary distros like Debian. (Some other binary distros, like openSUSE) also have centralized version control for package definitions.)

One way to conceptualize the qualitative differences in reproducibility outlined above is by examining the ways that Nix strengthens Debian's definition of reproducibility[4], which reads:

> A build is reproducible if given the same source code, build environment and build instructions,

For Nix, the build instructions can simply encode all of what Debian calls the ‘relevant attributes of the build environment’:

> Relevant attributes of the build environment would usually include dependencies and their versions, build configuration flags and environment variables as far as they are used by the build system (eg. the locale). It is preferable to reduce this set of attributes.

And similarly, for NixOS, the acquisition of source code is folded into the build instructions and the ‘build environment’ (i.e., caches being available or GitHub not being down). So every Nix package that is reproducible at all is reproducible in a more general way than a reproducible Debian package.

And NixOS/Nix have had to do real work to make their systems reproducible in ways that Debian is not. Unlike much of Debian's work its benefits can't really be shared with distros of a different design— but the converse is sometimes true as well. For example, Debian's work on rooting out non-determinism in package post-install hooks[5] is useless (and unnecessary) for NixOS, Guix, and Distri, since their packages don't have post-install hooks.

There are also lots of little ways that issues Debian has worked on either reflect the relative weakness of this notion of reproducibility (e.g., ‘All relevant information about the build environment should either be defined as part of the development process or recorded during the build process.’[6] is a way of saying ‘the build environment should be reproducible or merely documented’) or overcoming challenges that systems designed with reproducibility in mind from the start simply don't face.

At the same time, the Reproducible Builds website refers to publications[7] by former Nix developers who directly cite the original Nix paper from 2004, whereas Debian's effort didn't begin in earnest until 2013.[8]

Compared to the Nix community, Debian is huge. And they've leveraged their collective expertise and considerable volunteer force to do a ton of work toward reproducible builds which has benefited reproducibility for everyone, including NixOS. Doubtless every remotely attentive member of the Nix community is grateful for that work, which a small community like Nix's could hardly have taken up on its own. But Nix has been attacking reproducibility issues at a different level (reproducing build environments, source code, and whole systems (in terms of behavior, if not bits)) in a meaningful way since long before Debian's reproducible builds effort got going. And some of those efforts have informed the wider reproducible builds effort, just like some of Debian's efforts have not been applicable to every project in the F/OSS community which is interested in reproducible builds.

So: let's praise Debian loudly and often for their work here and be clear that NixOS' reproducibility couldn't be where it is today without that work... but let's also be clear that Nix/NixOS absolutely has blazed some trails in the territory of reproducibility— a terrain that both communities are still mapping out together. :)

—

1: https://michael.stapelberg.ch/posts/2019-03-10-debian-windin...

2: https://wiki.debian.org/ReproducibleInstalls/LiveImages

3: https://www.preining.info/blog/2014/06/debian-pristine-tar-p...

4: https://reproducible-builds.org/docs/definition/

5: https://reproducible-builds.org/docs/system-images/

6: https://reproducible-builds.org/docs/recording/

7: https://reproducible-builds.org/docs/publications/

8: https://wiki.debian.org/ReproducibleBuilds/History#Kick-off

Does this still matter if you can work your way up to a cross compiler, though? Do you actually need to go all the way down to ‘native’ hex monitors for a bunch of architectures or whatever?