Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You don’t need reproducible builds

blog.cmpxchg8b.com

Post reply on HN
Print view View original

91–100 of 179 posts

Re: You don’t need reproducible builds

#91

Post by joshuamorton »

jeffbee wrote: post #87
joshuamorton wrote: post #85

Earlier quoted context omitted.

As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.

Hrmm. Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested. Unit tests are generally fastbuild artifacts that are linked with many objects that are not in the release, including the test's main function and the test cases themselves. Integration tests and end-to-end tests often run with NDEBUG undefined and with things like sanitizers and checked a…

> Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested.

Of course.

> I would say that hardly anyone runs unit tests on release build artifacts just because it takes forever to produce them.

I don't know that this follows: just because 99% of the invocations of your unit test are in fastbuild doesn't mean that you don't also test everything in opt at least once.

Re: You don’t need reproducible builds

#92

Post by derriz »

D895n9o33436N42 wrote: post #8
mainguy wrote: post #4

I think OP is coming from a different perspective than I (corporate bespoke solution builder) do. When I say "reproducible build" I mean a build that is the same on any machine (i.e. no special magic necessary to build a "official" version of the code). Too often in corporate environments, getting a local build or setting up a new build pipeline involves arcane black magic and/or copy/pasting weird libraries that can…

For this reason I’ve been dockerizing my builds for almost five years. I was late to the Docker party, but when I saw the benefits it brings to build pipelines, I was sold. It's true that a dockerized build isn’t any simpler than its non-dockerized ancestor, but at least there’s a Dockerfile that lays bare all the black magic and special sauce which goes into each build. And it can be version controlled to watch for…

But docker builds themselves are generally not reproducible, so I don't really see the gain?

Yes with some effort, they can be made reproducible but the vast majority of the dockerfiles that I've encountered do not pin the versions of every dependency.

Some might pin a few key dependencies but nearly all do an apt/rpm/whatever update at some point followed by a bunch of install commands which don't specify versions.

Re: You don’t need reproducible builds

#93

Post by josh2600 »

Build reproducibility doesn't solve the problem of "how do I know the computer I don't control is actually running the program I think it is?"

The only solutions are related to fully homomorphic encryption, which is a way of running a program on a computer you don't control where the operator of that computer cannot learn information about the work it is performing. Unfortunately, pure software approaches to FHE haven't been proven out.

It is possible to get the benefits of FHE by utilizing a software enclave, but then you have a different trust narrative that involves hardware (and attestation services). The point of remote attestation is that you can verify that a computer you don't control is running a specific binary (verified by the hash and a signing key) at a specific moment in time. The problem is that the chain of trust for the signing key traces back to the manufacturer of the enclave.

The holy grail would be some way of decentralizing the trust of the enclave OR finding a way to do purely software based FHE that was fast. I'm not holding my breath for the latter, but the former might be possible in the next decade.

In short, if you have a reproducible build but you don't have a way of verifying that binary is actually running on a remote server you don't control, the reproducibility of the build is a moot point. I sorta think the only systems where build reproducibility matters today are ones that use enclaves (but I'm willing to be disabused of this notion if people feel otherwise).

Re: You don’t need reproducible builds

#94

Post by richardwhiuk »

Diggsey wrote: post #13

There are a lot of reasons to prefer reproducible builds, and many of them are not security related... It seems a bit presumptuous to argue that noone needs reproducible builds because one particular security argument is flawed. First, a non-flawed security argument: it only takes one non-malicious person to build a package from source and find that it doesn't match the distributed binary to spot a problem. Sure, if…

Agree.

In critical infrastructure, it's often essential to provide customers with a single fix. If you can't reproduce the build that they have, then you can't do that.

Re: You don’t need reproducible builds

#95

Post by jeffbee »

joshuamorton wrote: post #91
jeffbee wrote: post #87

Earlier quoted context omitted.

Hrmm. Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested. Unit tests are generally fastbuild artifacts that are linked with many objects that are not in the release, including the test's main function and the test cases themselves. Integration tests and end-to-end tests often run with NDEBUG undefined and with things like sanitizers and checked a…

> Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested. Of course. > I would say that hardly anyone runs unit tests on release build artifacts just because it takes forever to produce them. I don't know that this follows: just because 99% of the invocations of your unit test are in fastbuild doesn't mean that you don't also test everything in opt a…

I can't remember seeing any cc_test target at Google that ran with realistic release optimizations (AutoFDO/SamplePGO+LTO) and even if they did it's still not the release binary because it links in the test case and the test main function.

Re: You don’t need reproducible builds

#97

Post by perryizgr8 »

For us, the biggest benefit of reproducible builds is debuggability. One year later, if we get a bug report from a customer, we can fully recreate the issue in house, patch it, and provide a new build with confidence that we fixed the problem.

Without this, in a huge interconnected system you go insane trying to control all the variables needed to figure out what exactly went wrong.

Re: You don’t need reproducible builds

#98

Post by UncleEntity »

wildmanx wrote: post #72
rocqua wrote: post #35

So, we have source code for the signal app. We can audit that source code, to ensure no key-leakage occurs. Next, I install signal on my phone via the app store. How do I know the app I installed matches the source code that was audited? After all, google / Apple could decide / be forced to provide a modified binary. Reproducible builds work for that. Alternatively, consider debian. I ain't got time to compile every…

> Next, I install signal on my phone via the app store. How do I know the app I installed matches the source code that was audited? After all, google / Apple could decide / be forced to provide a modified binary. > Reproducible builds work for that. You read the original post, right? He discusses this at length. Actually, right in the beginning. In short, if you go through the dance of building the binary yourself to…

Someone, we'll call them Bob, goes to all the trouble to build the binary, finds out that it doesn't match with what the app store is distributing then just shrugs their shoulders and uses it to connect with other users in a secure messaging network because their binary is correct?

What could possibly go wrong?

Re: You don’t need reproducible builds

#99

Post by R0b0t1 »

mrob wrote: post #25
lucideer wrote: post #10

> Q. If a user has chosen to trust a platform where all binaries must be codesigned by the vendor, but doesn’t trust the vendor, then reproducible builds allow them to verify the vendor isn’t malicious. > I think this is a fantasy threat model. If the user does discover the vendor was malicious, what are they supposed to do? > The malicious vendor can simply refuse to provide them with signed security updates instead…

And vendor malice isn't the only reason why the software might be harmful. What if the vendor's toolchain was compromised without their knowledge? Reproducible builds provide a means for third parties to verify that it wasn't.

The holy grail of reproducible builds is achieving the same binary via different compilers. This was, at least when I started looking into reproducible builds, why I wanted to do it and why others wanted to do it. The other benefits are kind of side benefits.

Re: You don’t need reproducible builds

#100

Post by joshuamorton »

jeffbee wrote: post #95
joshuamorton wrote: post #91

Earlier quoted context omitted.

> Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested. Of course. > I would say that hardly anyone runs unit tests on release build artifacts just because it takes forever to produce them. I don't know that this follows: just because 99% of the invocations of your unit test are in fastbuild doesn't mean that you don't also test everything in opt a…

I can't remember seeing any cc_test target at Google that ran with realistic release optimizations (AutoFDO/SamplePGO+LTO) and even if they did it's still not the release binary because it links in the test case and the test main function.

Did you look in the CI system for configurations there? I see FDO enabled in those tests. (Speaking at a high level, configurations can be modified in bazelrc and with flags without being explicitly listed in the cc_test rule itself)

> release binary because it links in the test case and the test main function.

Sure, but it's verifiably the same object files as get put into the release artifact.

Post reply on HN