Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

discourse.nixos.org

Post reply on HN
Print view View original

151–160 of 179 posts

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#151

Post by TheDong »

TacticalCoder wrote: post #120
onedognight wrote: post #7

Rebuilding the minimal ISO from source is an impressive milestone on the journey to a system that builds from source reproducibly. Guix had an orthogonal but equally impressive milestone on the same journey recently[0], bootstrapping a full compiler toolchain from a single reproducible 357 byte binary without any other binary compiler blobs. These two features may one day soon be combined to reproducibly build a full…

That is amazing and it is great to see there are people out there fighting the good fight (while others ask: "but where's the benefit!? if there's a backdoor, everybody is still going to get the backdoor!" ). > it gives us a reliable way to verify the binaries we ship are faithful to their sources That's the thing many don't understand: it's not about proving that the result is 100% trustable. It's about proving it's…

To me, the largest benefit isn't even related to "bad guys", but rather in being able to understand and debug issues.

Reproducibility makes bugs more shallow. If hydra builds a bit-for-bit identical iso to what you build locally, that means a developer can make a change to the iso inputs, test it, and know that testing will also apply to the final ci-built one.

If a user reports a bug in the iso, and you want to test if a change fixes it locally, you can start from an identical source-code commit as the iso was built from, make some minimal changes, and debug from there, all without worrying that you're accidentally introducing unintended differences.

It minimizes "but it works on my machine" type issues.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#152

Post by raboof »

watersucks wrote: post #127
Reventlov wrote: post #4

For those wondering : it should be remembered that the reproducibility of Nix / NixOS / Nixpkgs is only a reproducibility of the sources: if the sources change, one is warned, but it is not a question of the reproducibility of the binaries (which can change at each build). This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically. Guix, Archlinux, Debian do the bina…

Doesn’t the content-addressed derivation experimental feature address this issue? Instead of store hashes being input-addressed as you mention, the derivation outputs are used to calculate the store hash, which ensures binary reproducibility.

Not really:

With input-addressing you look things up in the store based on the input hash. You can determine the input hash yourself, but you have to trust the store to provide a response that corresponds to the sources. With Reproducible Builds you can have a third party confirm that output matches that input.

With content-addressing, you look things up in the store based on the output hash. You no longer need to trust the store here: you can check yourself the response matches the hash. However, you now have to trust who-ever told you that output hash corresponds to the input you're interested in. With Reproducible Builds you can now have a third party confirm that output hash matches that input.

I have not worked with content-addressed nix in depth yet, but my understanding is that this stores the mapping between the inputs and their output hashes in 'realizations' which are also placed in the store. Reproducible Builds will still be useful to validate this mapping is not tampered with.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#153

Post by raboof »

Uptrenda wrote: post #125

Wouldn't this help solve the problem Ken Thompson wrote about in 'reflections on trusting trust?' If you can fully bootstrap a system from source code then it's harder to have things like back-doored compilers.

It indeed helps, but it is not a full 'solution': you could still in theory envision elaborate backdoors in the 'environment' in which the ISO is built. If you really want to 'solve' the problem describe there, you could look into Diverse Double Compiling (https://dwheeler.com/trusting-trust/) or bootstrapping the entire environment (https://bootstrappable.org/) - see also the 'Aren’t there bootstrap problems with the above approach?' section of the post.

Reproducing the build already goes a long way in making such attacks increasingly unlikely, though.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#154

Post by raboof »

Reventlov wrote: post #99
clhodapp wrote: post #26

Earlier quoted context omitted.

That is not true at all, with respect to the aims or the reality of nixpkgs. The original post here is talking about reproducing the (binary) minimal iso, which contains a bunch of binary packages.

It is true. The original post writes about reproducing the minimal iso, which contains probably around 1% of the packages in nixpkgs. The remaining packages are not tested regarding binary reproducibility, or, at least, not in a systematic manner, which means regressions may happen regularly (which is exactly what happened with the .iso, see the previous announcement from 2021: https://discourse.nixos.org/t/nixos-uns…

While I would love testing reproducibility more systematically, it would not really have helped for the Python 3.10 regression: in this case we knew full well that it would break reproducibility even before we merged the change, but the performance advantage it unlocked seemed too big to ignore. Such trade-offs are luckily rare - I'm happy that with Python 3.11 we can now have both :)

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#155

Post by lifeisstillgood »

k8svet wrote: post #88
lifeisstillgood wrote: post #19

Earlier quoted context omitted.

I am on a similar journey I built https://github.com/mikadosoftware/workstation (hey nearly 500 stars!) as the idea of defining a reproducible laptop build. I don't think docker is the right level - so my next project when i have free time (!) is to do a box build that then might compile to docker I think there is a sensible point of being able to define via nix both developer workstations and servers

Except it's Docker, and like virtually all Dockerfiles, it immediately runs "apt-get update", tossing reproducibility out the window.

yes, hence the (planned) move to flake.nix and then compile to docker.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#156

Post by lrvick »

__MatrixMan__ wrote: post #148
lrvick wrote: post #144

Earlier quoted context omitted.

I totally grant that the nix system is good at making sure that a given git commit produces a given result. They do a fantastic job at everything from miles 2-10. They just skip the first mile of integrity, which is also one of the easiest to tamper with. 9/10 popular package developers I audit have SMS as a backup account recovery method on their email accounts. Easy to see when I try to reset their email passwords…

You're talking sense, but this is due diligence for a developer, not for an operating system or a package manager. You're free to map your package definitions to the commits they contain and verify any signatures that you find there, but that process will have nothing to do with whether other people have signed the instructions that your machine follows to fetch and build the contents of that commit.

Sure I could write a lot of tooling to try and do this sort of basic verification manually in Nix every single time i pin a new package, but at that point why am I using nix over distros that have native support for maintainer level supply chain integrity?

Compare to the Arch model where all official packages must be signed with keys belonging to a reasonably well vetted web of trust.

Developers outside the web of trust that want to contribute yolo unsigned packages to Arch still can, but those must go into AUR where users must opt-in to and manually review each individual untrusted/unsigned package.

https://wiki.archlinux.org/title/Pacman/Package_signing

Nix decided to have the yolo AUR model by default, with no method to elect to use only signed packages, because signing and web of trust are not even supported at all, even optionally.

This is wildly irresponsible given how many people use Nix today.

Nix is two steps forward in deterministic, immutable, and unprivileged package management, and one giant leap backwards in supply chain integrity.

This is why we cannot have nice things.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#157

Post by Smaug123 »

__MatrixMan__ wrote: post #149
amarshall wrote: post #10

Earlier quoted context omitted.

> NixOS uses "reproducible" to mean "with the same Nix code, you get the same program behaviour". Note that ”Nix code” also includes the hashes of all non-Nix sources. One way to think of it is that Nix has reliable build cache invalidation. > This is more/less what people hope Dockerfiles provide. Indeed, but importantly they do not provide input-reproducibility (while Nix does) because, at least, there are no hashe…

PSA: you can build OCI images with nix, then they'll be a pure function of their input like we've all wished was the case with Dockerfiles.

(And Nix derivations compose, whereas Dockerfiles entirely do not.)

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#158

Post by hiepph »

KennyFromIT wrote: post #16

I've lived in the Red Hat ecosystem for work recently. How does this compare to something like... Fedora Silverblue? Ansible? Fedora Silverblue + Ansible?

Ansible makes mutable changes to the OS, task by task.

Nix is immutable. A new change is made entirely new, and only after the build is successful, all packages are "symlinked" to the current system.

Fedora Silverblue is based on ostree [1]. It works similarly like git, but on your root tree. But it requires you to reboot the whole system for the changes to take effect. Since Nix is just symlinked packages, you don't need to reboot the system.

More detailed explanation here [2].

[1]: https://github.com/ostreedev/ostree

[2]: https://dataswamp.org/~solene/2023-07-12-intro-to-immutable-...

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#159

Post by bloak »

zbentley wrote: post #147
bloak wrote: post #104

Earlier quoted context omitted.

Much as I like Git, I'm not sure I like the idea of the artefacts depending on the git commit and therefore on the entire git history. I rather feel the artefacts should only depend on the actual source and not on a particular version control system used for storing the source.

You're welcome to include full sources, or not-tied-to-git directions to acquire them, with your release binaries. Regardless, whether or not you do that is a discussion of distribution format, not binary reproducibility. Your distribution can contain as much (or as little) additional material as you like along with your release binaries.

Absolutely. I'm just stating my personal preference, that's all.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#160

Post by KennyFromIT »

hiepph wrote: post #158
KennyFromIT wrote: post #16

I've lived in the Red Hat ecosystem for work recently. How does this compare to something like... Fedora Silverblue? Ansible? Fedora Silverblue + Ansible?

Ansible makes mutable changes to the OS, task by task. Nix is immutable . A new change is made entirely new, and only after the build is successful, all packages are "symlinked" to the current system. Fedora Silverblue is based on ostree [1]. It works similarly like git, but on your root tree. But it requires you to reboot the whole system for the changes to take effect. Since Nix is just symlinked packages, you don'…

This is a great explanation of the technical differences between the available options. What are the practical differences - is one option better from a maintenance and usability standpoint for creating systems that are reproducible?
Post reply on HN