Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

discourse.nixos.org

Post reply on HN
Print view View original

121–130 of 179 posts

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#121

Post by prateem_ »

Foxboron wrote: post #117
prateem_ wrote: post #115

Earlier quoted context omitted.

I am probably misunderstanding your point BUT I have actually depended on Nix for "reproducible docker images" for confidential compute usecase so that all parties can independently verify the workload image hash. Rarely (actually only once) it did fail to produce bit identical images every other time it successfully produced bit identical images on very different machine setups. Granted this is not ISO but docker im…

I'm very sure you are actually just rebuilding the container images themselves, not the package tree you are depending on. Building reproducible ISOs, or container images, with a package repository as a base isn't particularly hard these days.

I see what you mean. Thanks for clarifying. Even so, Nix is no worse placed than those other distributions for bit reproducibility. Correct?

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#122

Post by TacticalCoder »

kaba0 wrote: post #34
ahmedfromtunis wrote: post #32

Stupid question as I never worked on something like this before: why isn't reproducibility the default behavior? I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time? I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.

Parallelism. There might be actions that are not order-independent, and the state of the CPU might result in slightly different binaries, but all are correct.

> There might be actions that are not order-independent, and the state of the CPU might result in slightly different binaries, but all are correct.

Well no: that's really the thing reproducible packages are showing: there's only one correct binary.

And it's the one that's 100% reproducible.

I'd even say that that's the whole point: there's only one correct binary.

I'll die on the hill that if different binaries are "all correct", then none are: for me they're all useless if they're not reproducible.

And it looks like people working on entire .iso being fully bit-for-bit reproducible are willing to die on that hill too.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#123

Post by kaba0 »

TacticalCoder wrote: post #122
kaba0 wrote: post #34

Earlier quoted context omitted.

Parallelism. There might be actions that are not order-independent, and the state of the CPU might result in slightly different binaries, but all are correct.

> There might be actions that are not order-independent, and the state of the CPU might result in slightly different binaries, but all are correct. Well no: that's really the thing reproducible packages are showing: there's only one correct binary. And it's the one that's 100% reproducible. I'd even say that that's the whole point: there's only one correct binary. I'll die on the hill that if different binaries are "…

See my reply to the sibling post — binary reproducibility is not the end goal. It is an important property, and I do agree that most compiler toolchains should strive for that, but e.g. it might not be a priority for, say, a JIT compiler.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#124

Post by Foxboron »

prateem_ wrote: post #121
Foxboron wrote: post #117

Earlier quoted context omitted.

I'm very sure you are actually just rebuilding the container images themselves, not the package tree you are depending on. Building reproducible ISOs, or container images, with a package repository as a base isn't particularly hard these days.

I see what you mean. Thanks for clarifying. Even so, Nix is no worse placed than those other distributions for bit reproducibility. Correct?

It's unclear at the moment because of the limited testing (minimal ISO and a Gnome ISO) vs Arch/Debian/Guix rebuilding entire package repositories.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#126

Post by tikhonj »

londons_explore wrote: post #101
beisner wrote: post #80

Earlier quoted context omitted.

if you have a reproducible build, then the notion of "software was built on date by user " is kind of useless information, no? Because it does not matter - if you can verify that a specific git hash of a codebase results in a particular binary through reproducible builds, a malicious adversary could have built it yesterday and given it to me and i can be almost surely confident (barring hash-collisions...) it's ident…

The usecase is: user wants an easy way to know, from the GUI of some running software, exactly what build/version/git commit/branch/date they're running - perhaps to file a bug report for example. The actual build date doesn't matter if the software is reproducible - but its a proxy for 'how out of date is this software'.

If you actually had reproducible builds, the build date would not tell you anything about how out of date the software is—you would only need the date of the source code the binary was built from. By definition, the binary you'd get from building a version of the source today would be identical to the version you'd get building it the day that version of the source was finished.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#127

Post by watersucks »

Reventlov wrote: post #4

For those wondering : it should be remembered that the reproducibility of Nix / NixOS / Nixpkgs is only a reproducibility of the sources: if the sources change, one is warned, but it is not a question of the reproducibility of the binaries (which can change at each build). This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically. Guix, Archlinux, Debian do the bina…

Doesn’t the content-addressed derivation experimental feature address this issue? Instead of store hashes being input-addressed as you mention, the derivation outputs are used to calculate the store hash, which ensures binary reproducibility.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#128

Post by Smaug123 »

watersucks wrote: post #127
Reventlov wrote: post #4

For those wondering : it should be remembered that the reproducibility of Nix / NixOS / Nixpkgs is only a reproducibility of the sources: if the sources change, one is warned, but it is not a question of the reproducibility of the binaries (which can change at each build). This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically. Guix, Archlinux, Debian do the bina…

Doesn’t the content-addressed derivation experimental feature address this issue? Instead of store hashes being input-addressed as you mention, the derivation outputs are used to calculate the store hash, which ensures binary reproducibility.

Ish. This is covered in section 6.4.1 of Eelco's thesis (https://edolstra.github.io/pubs/phd-thesis.pdf). It all becomes much simpler if evaluating a build many times can only ever result in one output, but the Nix content-addressed model does permit multiple outputs. In such cases, the system just has to choose a canonical output and use that one, rewriting hashes as necessary to canonicalise inputs which are non-canonical.

Re: NixOS Reproducible Builds: minimal ISO successfully independently rebuilt

#129

Post by lmm »

ralferoo wrote: post #110
rssoconnor wrote: post #102

Earlier quoted context omitted.

The program does also dispose of comment lines. One could argue that this is just a kind of trick so they can say the next "binary" is actually a "source" file because it happens to be written by a human in ASCII. Still the phase distinction between what is a source and what is a binary becomes blurry at this low level. I believe the next stage of compiling is to, writing in ASCII represented machine code with commen…

Yeah, I kind of agree, but my issue is kind of with this statement (in the link from the peer post): > What if we could bootstrap our entire system from only this one hex0 assembler binary seed? We would only ever need to inspect these 500 bytes of computer codes. Every later program is written in a more friendly programming language: Assembly, C, … Scheme. And my issue is that this isn't true. hex1 isn't written in…

> Having hex0 doesn't add anything to the mix, other than being shorter than hex1, because you still have the same initial bootstrap problem of how you can prove that the hex0 binary represents the hex in its source vs the hex1 binary and its source

Well presumably you toggle hex0 in on the front panel and then type hex1 with the keyboard, which is easier than toggling in the binary of hex1.

Post reply on HN