Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

discourse.nixos.org

Post reply on HN
Print view View original

61–70 of 232 posts

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#61

Post by peterkelly »

avalys wrote: post #2

Can anyone comment on the significance of this accomplishment, and why it was hard to achieve before? I (naively, apparently) assumed this had been possible with open-source toolchains for a long time.

For some reason, many compilers and build scripts have traditionally been written in a way that's not referentially transparent (a pure function from input to output). Unnecessary information like the time of the build, absolute path names of sources and intermediate files, usernames and hostnames often would find their way into build outputs. Compiling the same source on different machines or at different times would yield different results.

Reproducible builds avoid all this and always produce the same outputs given the same inputs. There's no good reason (that I can think of) why this shouldn't have been the case all along, but for a long time I guess it just wasn't seen as a priority.

The benefit of reproducible builds is that it's possible to verify that a distributed binary was definitely compiled from known source files and hasn't been tampered with, because you can recompile the program yourself and check that the result matches the binary distribution.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#62

Post by bigiain »

initplus wrote: post #58
dcposch wrote: post #11

This really deserves more love. Who remembers Ken Thompson's "Reflections on Trusting Trust"? The norm today is auto-updating, pre-built software. This places a ton of trust in the publisher. Even for open-source, well-vetted software, we all collectively cross our fingers and hope that whoever is building these binaries and running the servers that disseminate them, is honest and good at security. So far this has mo…

Building from source doesn't have to be inaccessible, if the build tooling around it is strong. Modern compiled languages like Go (or modern toolchains on legacy languages like vcpkg) have a convention of building everything possible from source. So at least for software libraries building from source is definitely viable. Fro end user applications it's another story though, doubt we will ever be at a point where bui…

Building from source also doesn’t buy you very much, if you haven’t inspected/audited the source.

The upthread hypothetical of a compromised package manager equally applies to a compromised source repo.

_Maybe _ you always check the hashes? _Maybe_ you always get the hashes from a different place to the code? _Maybe_ the hypothetical attacker couldn’t replace both the code you download and the hash you use to check it?

(And as Ken pointed out decades ago, maybe the attacker didn’t fuck with your compiler so you had lost before you even started.)

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#63

Post by Foxboron »

mjg59 wrote: post #53
radicalcentrist wrote: post #36

Earlier quoted context omitted.

Reproducibility is necessary, but unfortunately not sufficient, to stop a "Trusting Trust" attack. Nixpkgs still relies on a bootstrap tarball containing e.g. gcc and binutils, so theoretically such an attack could trace its lineage back to the original bootstrap tarball, if it was built with a compromised toolchain.

Diverse double compilation should allow a demonstration that the toolchain is trustworthy.

Indeed, and with the work done by Guix and the Reproducible Builds project we do have a real-world example of diverse double compilation which is not just a toy example utilizing the GNU Mes C compiler.

https://dwheeler.com/trusting-trust/#real-world

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#64

Post by bigiain »

zamadatix wrote: post #38
jnxx wrote: post #32

Earlier quoted context omitted.

No, it is not always the maintainer. Imagine you download a binary software package via HTTPS. In theory, the integrity of the download is protected by the server certificate. However, it is possible that certificates get hacked, get stolen, or that nation states force CAs to give out back doors. In that case, your download could have been changed on the fly with arbitrary alterations. Reproducible builds make it pos…

Same as when you download the source instead of the binary and see it reproducibly builds the backdoored binary. And at this point we're back to "Build from source. This will always be a deeply niche thing to do. It's slow, inconvenient, and inaccessible except to nerds." anyways. It's not that reproducible builds provide 0 value it's that they don't truly solve the trust problem as initially stated. They also have n…

I guess reproducible builds solve some of the problems in the same way TLS/SSL solves some of the problems.

Most of the world is happy enough with the soft guarantee of: “This is _probably_ your bank’s real website. Unless a nation state is misusing their control over state owned certificate authorities, or GlobalSign or LetsEncrypt or whoever has been p0wned.”

Expecting binary black and white solutions to trust problems isn’t all that useful, in my opinion. Often providing 50% more “value” in trust compared to the status quo is extremely valuable in the bigger picture.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#65

Post by aseipp »

fouronnes3 wrote: post #6

Are there synergies with the Debian reproducible build project that this can benefit from?

Debian has been a major driver in making many pieces of software reproducible across every distribution; that Debian maintainers so often submit patches upstream and work directly to solve these issues is a big reason for this.

In other words: the work Debian has done absolutely set the stage for this to happen, and it would have taken much longer without them.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#66

Post by squiggleblaz »

zamadatix wrote: post #38
jnxx wrote: post #32

Earlier quoted context omitted.

No, it is not always the maintainer. Imagine you download a binary software package via HTTPS. In theory, the integrity of the download is protected by the server certificate. However, it is possible that certificates get hacked, get stolen, or that nation states force CAs to give out back doors. In that case, your download could have been changed on the fly with arbitrary alterations. Reproducible builds make it pos…

Same as when you download the source instead of the binary and see it reproducibly builds the backdoored binary. And at this point we're back to "Build from source. This will always be a deeply niche thing to do. It's slow, inconvenient, and inaccessible except to nerds." anyways. It's not that reproducible builds provide 0 value it's that they don't truly solve the trust problem as initially stated. They also have n…

The question isn't whether they're perfect, nor is it whether they prevent anything. But it does help a person who suspects something is up rule certain things in and out, which increases the chances that the weak link can be found and eliminated.

If you have a fair suspicion that something is up and you discover that when you compile reproduceable-package you get a different output than when you download a prebuilt reproduceable-package, you've now got something to work with.

Your observation that they don't truly solve the trust problem is true. But it's somehow not relevant. It is better to be better off.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#67

Post by aseipp »

jeroenhd wrote: post #47
amelius wrote: post #43

Earlier quoted context omitted.

What would you recommend instead of NVidia's Jetson embedded platform?

I'm not familiar with the market the Jetson is in and what purposes it serves. From a quick Google, it seems to build boards for machine learning? If that's true, I'm pretty sure Google and Intel have products in that space, and I'm sure there's other brands I don't know of. If Nvidia has its own distribution, it might well work for as long as it's willing to maintain the software because then they can tune their ope…

Google and Intel's solutions are just as proprietary, with the downside almost nobody uses them so bugs, performance, supported tooling, community, and support windows are often much worse. It's not even clear their solutions actually offer better performance in general, given this. (And if you think proprietary Nvidia software packages are infuriating messes, wait until you try Intel proprietary software.) How you feel about their history of Linux support all that said is basically irrelevant, and they'll continue to dominate because of it.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#69

Post by tbrock »

dcposch wrote: post #11

This really deserves more love. Who remembers Ken Thompson's "Reflections on Trusting Trust"? The norm today is auto-updating, pre-built software. This places a ton of trust in the publisher. Even for open-source, well-vetted software, we all collectively cross our fingers and hope that whoever is building these binaries and running the servers that disseminate them, is honest and good at security. So far this has mo…

Why does building from source help? It’s not like people are reading every line of the source before building it anyway 99.99% of the time.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#70

Post by 1vuio0pswjnm7 »

dcposch wrote: post #11

This really deserves more love. Who remembers Ken Thompson's "Reflections on Trusting Trust"? The norm today is auto-updating, pre-built software. This places a ton of trust in the publisher. Even for open-source, well-vetted software, we all collectively cross our fingers and hope that whoever is building these binaries and running the servers that disseminate them, is honest and good at security. So far this has mo…

"- Build from source. This will always be a deeply niche thing to do. It's slow, inconvenient, and inaccessible except to nerds."

I prefer compiling from source to binary packages. For me it is neither slow, incovenient nor inaccessible.

Only with larger, more complex programs does compiling from source become a PITA.

The "solution" I take is to prefer smaller, less complex programs over larger, more complex ones.

If I cannot compile a program from source relatively quickly and easily, I do not voluntarily choose it as a program to use daily and depend on.

For compiling OS, I use NetBSD so perhaps I am spoiled because it is relatively easy to compile.

That said, I understand the value of reproducible builds and appreciate the work being done on such projects.

Post reply on HN