Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

discourse.nixos.org

Post reply on HN
Print view View original

211–220 of 232 posts

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#211

Post by Gravyness »

londons_explore wrote: post #184
dcposch wrote: post #11

This really deserves more love. Who remembers Ken Thompson's "Reflections on Trusting Trust"? The norm today is auto-updating, pre-built software. This places a ton of trust in the publisher. Even for open-source, well-vetted software, we all collectively cross our fingers and hope that whoever is building these binaries and running the servers that disseminate them, is honest and good at security. So far this has mo…

> and 6mo later every computer ... gets ransomwared. I'm really surprised such an attack hasn't happened already. It seems so trivial for a determined attacker to take over an opensource project (plenty of very popular projects have just a single jaded maintainer). The malicious compiler could inject an extra timed event into the main loop for the time the attack is scheduled to begin, but only if it's >3 hours away,…

> I'm really surprised such an attack hasn't happened already.

If you count npm packages this happened quite a few times already. People (who don't understand security very well) seems to be migrating to python now.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#212

Post by therealjumbo »

pjmlp wrote: post #183
squiggleblaz wrote: post #182

Earlier quoted context omitted.

So? We are trying to protect against a malicious interloper damaging the machine of a trusted and trustworthy partner. You are bringing up red herrings about trusted partners being malicious and untrustworthy. Do you genuinely believe we should only solve a problem if it leads to a perfect outcome?

I genuinely believe to spend resources on issues where ROI is positive. So far exploits on FOSS kind of prove the point not everyone is using Gentoo, reading every line of code on their emerged packakges, let alone similar computing models. Now if we are speaking about driving the whole industry to where security bugs, caused by using languages like C that cannot save us from code reviews unless done by ISO C languag…

> I genuinely believe to spend resources on issues where ROI is positive.

How are you measuring the ROI of security efforts inside an OSS distro like debian or nixos? The effort in such orgs is freely given, so nobody knows how much it costs. And how would you calculate the return on attacks that have been prevented? Even if an attack wasn't prevented you don't know how much it cost, and you might not even know if it happened (or if it happened due to a lapse in debian.)

>So far exploits on FOSS kind of prove the point not everyone is using Gentoo, reading every line of code on their emerged packakges, let alone similar computing models.

Reproducible builds is attempting to mitigate a very specific type of attack, not all attacks in general. That is, it focuses on a specific threat model and countering that, nothing else. It's not a cure for cancer either.

>Now if we are speaking about driving the whole industry to where security bugs, caused by using languages like C that cannot save us from code reviews unless done by ISO C language lawyers and compiler experts in UB optimizations, are heavily punished like construction companies are for a fallen bridge, then that would be interesting.

This is just a word salad of red herrings. Different people can work on different stuff at the same time.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#213

Post by Ericson2314 »

mikepurvis wrote: post #107
esjeon wrote: post #96

Earlier quoted context omitted.

> I'm grateful to the nixos team for being beating a trail thru the jungle here. Retrofitting reproducibility onto a big software project that grew without it, is hard work. Actually, it's Debian guys who pushed reproducible build hard in the early days. They upstreamed necessary changes and also spread the concept itself. This is a two-decade long community effort. In turn, NixOS is mostly just wrapping those projec…

I think both efforts have been important and have benefitted each other. Nix has always had purity/reproducibility as tenets, but indeed it was Debian that got serious about it on a bit-for-bit basis, with changes to the compilers, tools like diffoscope, etc. The broader awareness and feasibility of reproducible builds then made it possible for Nix to finally realise the original design goal of a content-addressed ra…

> where you don't need to actually sign your binary cache, but rather just sign a mapping between input hashes and content hashes.

Though you can and should sign the mapping!

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#214

Post by Ericson2314 »

taviso wrote: post #208
londons_explore wrote: post #185

Earlier quoted context omitted.

Where the dependency chain is long, this substantially reduces build work during development too. I'd guess that more than half of the invocations of gcc done by Make for example end up producing the exact same bit for bit output as some previous invocation.

I would point out that is literally what ccache (and Google goma) does, but doesn't require deterministic builds. Instead, it records hashes of preprocessed input and compiler commandlines. They don't make any security claims about this, it's just for speeding up builds.

What we currently do --- hashing inputs --- is the same ccache way. We just don't yet sandbox with the granularity yet.

What we want to id hash outputs. Say I replace 1 + 2 with 0 + 3. That will cause ccache to rebuild. We don't want downstream stuff to also be rebuilt. C-linking withing a package is nice in parallelizable, but in the general case there is more dependency chains and now that sort of thing starts to matter.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#215

Post by mikepurvis »

Ericson2314 wrote: post #213
mikepurvis wrote: post #107

Earlier quoted context omitted.

I think both efforts have been important and have benefitted each other. Nix has always had purity/reproducibility as tenets, but indeed it was Debian that got serious about it on a bit-for-bit basis, with changes to the compilers, tools like diffoscope, etc. The broader awareness and feasibility of reproducible builds then made it possible for Nix to finally realise the original design goal of a content-addressed ra…

> where you don't need to actually sign your binary cache, but rather just sign a mapping between input hashes and content hashes. Though you can and should sign the mapping!

Of course, yes— that was what I was saying. But the theory with content-addressability is that unlike a conventional distro where the binaries must all be built and then archived and distributed centrally, Nix could do things like age-out the cache and only archive the hashes, and a third party could later offer a rebuild-on-demand service where the binaries that come out of it are known to be identical to those which were originally signed. A similar guarantee is super useful when it comes to things like debug symbols.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#216

Post by taviso »

eru wrote: post #193
taviso wrote: post #176

Earlier quoted context omitted.

> With the reproducible build, you can start using the untrusted binary while you are still building your trusted one. That's not how it works, you have to reproduce it before it becomes trusted. > You can also have ten people on the internet verify the untrusted binary. Sure, then we have to build a complex consensus system that introduces a bunch of unsolved problems. My opinion is that this just isn't worth it, th…

> That's not how it works, you have to reproduce it before it becomes trusted. Eh, there's stuff you can do with software before you trust it. Eg you can start pressing the CDs or distributing the data to your servers. Just don't execute it, yet. > Sure, then we have to build a complex consensus system that introduces a bunch of unsolved problems. My opinion is that this just isn't worth it, there is practically noth…

> People don't do kernel reviews themselves. They just use the official kernel, and when someone finds a bug (or spots otherwise bad code), they notify the community.

There's a big difference here. When a vulnerability is found in the Linux kernel, that doesn't mean that you were compromised.

If a build was found to be malicious, then you definitely were compromised and it's little solace that it was discovered after the fact. This is why package managers check the deb/rpm signature before installing the software, not after.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#217

Post by zamadatix »

tadfisher wrote: post #57
zamadatix wrote: post #41

Earlier quoted context omitted.

Talos has 44 core/176 thread server options which can take 2 TBs of DDR4 that are FSF certified. The board firmware is also open and has reproducible builds.

That is way more expensive than a 16-core desktop, though. Workstations are a class above consumer-grade desktops and that's reflected in the price.

Talos have as low as 8 core desktop options as well this is just an example of how far you can take FLOSS hardware. Not that I consider a 16 core x86 desktop "consumer-grade" in the first place (speaking as a 5950X owner).

Probably not fit for replacing Grandma's budget PC but then again grandma probably isn't worried about the ARM cross compile performance of their machine running NixOS either.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#218

Post by Accujack »

dcposch wrote: post #11

This really deserves more love. Who remembers Ken Thompson's "Reflections on Trusting Trust"? The norm today is auto-updating, pre-built software. This places a ton of trust in the publisher. Even for open-source, well-vetted software, we all collectively cross our fingers and hope that whoever is building these binaries and running the servers that disseminate them, is honest and good at security. So far this has mo…

>The norm today is auto-updating, pre-built software.

Only if you define "norm" as what's prevalent in consumer electronics and phones. Certainly, if you go by numbers, it's more common than anything else.

That's not due to choice, though, it's because of the desires of corporations for ever more extensive control of their revenue streams.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#219

Post by Gaelan »

dcposch wrote: post #120
esjeon wrote: post #96

Earlier quoted context omitted.

> I'm grateful to the nixos team for being beating a trail thru the jungle here. Retrofitting reproducibility onto a big software project that grew without it, is hard work. Actually, it's Debian guys who pushed reproducible build hard in the early days. They upstreamed necessary changes and also spread the concept itself. This is a two-decade long community effort. In turn, NixOS is mostly just wrapping those projec…

Has a full linux image--something you can actually boot--existed as a reproducible build before today?

I have to imagine it's been done, at least with some stripped-down kernel+busybox situation. Not sure, though.

Re: Nixos-unstable’s ISO_minimal.x86_64-Linux is 100% reproducible

#220

Post by pxc »

microtonal wrote: post #180
taviso wrote: post #77

I don't see a single comment doubting the value of reproducibility, so I'll be the resident skeptic :) I think build reproducibility is a cargo cult. The website says reproducibility can reduce the risk of developers being threatened or bribed to backdoor their software, but that is just ridiculous. Developers have a perfect method for making their own software malicious: bugdoors. A bugdoor (bug + backdoor) is a del…

I think build reproducibility is a cargo cult. Most people here are debating you on the security angle, but in the case of Nix (and Guix) there is another important angle - reproducible builds make a content-addressed store possible. In Nix, the store is traditionally addressed by the hash of the derivation (the recipe that builds the package). For example, lr96h... in the path /nix/store/lr96h3dlny8aiba9p3rmxcxfda0i…

Another non-security angle: doesn't computer science also face a kind of replicability crisis related to the ability to acquire and compile source code associated with some published papers? Reproducible builds directly address that.

And it seems like even when that problem is resolved for the empirical component of computer science, bit-identical reproducibility could be valuable in case binaries are never submitted or distributed. This NixOS release is in a way a benchmark for how far we can currently get on a 'useful' system with that kind of reproducibility.

Post reply on HN