81–90 of 179 posts
Yes you do. How else can you trust an open source bitcoin client in the play store is the same version published on github?
Here's a hypothetical but a realistic scenario that not having reproducible builds is a major issue: 1. I create an app for a client. 2. Client installs the app on their 10,000 enterprise mobile devices and trains the users. 3. Month passes. I've changed everything in the app. 4. Major security or outage event happens. I need to change a line in the app and those 10,000 devices need to be updated ASAP. 5. If I can't…
You would still have to do it with reproducible builds. It's only reproducible if you have the same code. If you change something there is no reasons to get the same result, a bit like you would expect with a hash, it is deterministic, but that's all.
I think OP is coming from a different perspective than I (corporate bespoke solution builder) do. When I say "reproducible build" I mean a build that is the same on any machine (i.e. no special magic necessary to build a "official" version of the code). Too often in corporate environments, getting a local build or setting up a new build pipeline involves arcane black magic and/or copy/pasting weird libraries that can…
It’s a well-defined term though https://en.wikipedia.org/wiki/Reproducible_builds
The GNU project used reproducible builds in the early 1990s. Changelogs from 1992 indicate the ongoing effort. [4]
One of the older[5] projects to promote reproducible builds is the Bitcoin project with Gitian. Later, in 2013, the Tor (anonymity network) project started using Gitian for their reproducible builds.[6]
In July 2013 on the Debian project started implementing reproducible builds across its entire package archive.[7][8]
By July 2017 more than 90% of the packages in the repository have been proven to build reproducibly.[9]
In November 2018, the Reproducible Builds project joined the Software Freedom Conservancy.[10]
F-droid uses reproducible builds to provide a guarantee that the distributed APKs use the claimed free source code.[11
There are a lot of reasons to prefer reproducible builds, and many of them are not security related... It seems a bit presumptuous to argue that noone needs reproducible builds because one particular security argument is flawed. First, a non-flawed security argument: it only takes one non-malicious person to build a package from source and find that it doesn't match the distributed binary to spot a problem. Sure, if…
Earlier quoted context omitted.
If that's really what you are doing, then you are doing it horribly wrong.
You test prod builds?
Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
Earlier quoted context omitted.
You test prod builds?
As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
But if you're building client software artifacts, to unit test or integration test involves building different software in a different configuration, with different software and running it in a test harness. To facilitate unit testing or integration testing client software you:
- Build with a lower optimization level (-O0 usually) so that the generated code bares even a passing resemblance to what you actually wrote and your debugger can follow along.
- Generate debug info.
- Avoid stripping symbols.
- Enable logging.
- Build and link your tests code into a library artifact.
- Run it in a test harness.
That's not testing what you ship. It's testing something pretty close, obviously, but does not bear any semblance to a deterministic build.
Earlier quoted context omitted.
You test prod builds?
As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
Earlier quoted context omitted.
As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
I suppose we're referring to different kinds of testing. Manual QA, etc, on prod sure. But if you're building client software artifacts, to unit test or integration test involves building different software in a different configuration, with different software and running it in a test harness. To facilitate unit testing or integration testing client software you: - Build with a lower optimization level (-O0 usually)…
If you have test failures in opt/stripped mode, they're more annoying to debug yes, but wouldn't you want to know?
Another way of putting this is that when you
> - Build and link your tests code into a library artifact.
You build and link the same object files that will be built into the release binary artifact, deterministically.
> - Enable logging.
I, uhh, usually do this in my released software too.
Earlier quoted context omitted.
As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
Hrmm. Surely the vast majority of testing happens on non-release builds, despite the fact that release builds may also be tested. Unit tests are generally fastbuild artifacts that are linked with many objects that are not in the release, including the test's main function and the test cases themselves. Integration tests and end-to-end tests often run with NDEBUG undefined and with things like sanitizers and checked a…
Earlier quoted context omitted.
As part of the release process, yes...absolutely. Compared to basically every other part of release qualification (manual QA, canarying, etc.) re-testing on the prod build is so unbelievably cheap there's no reason to not.
I suppose we're referring to different kinds of testing. Manual QA, etc, on prod sure. But if you're building client software artifacts, to unit test or integration test involves building different software in a different configuration, with different software and running it in a test harness. To facilitate unit testing or integration testing client software you: - Build with a lower optimization level (-O0 usually)…
It's true that not all tests which are possible to run in debug configuration can also be run on a release artifact; e.g. if there are test-only interfaces that are compiled out in the release configuration.
I think maybe the source of the confusion in this conversation is perhaps the kind of artifact being tested? For example, if I were developing ffmpeg, to choose an arbitrary example, I would absolutely have tests which operate on the production artifact -- the binary compiled in release mode -- which only exercise public interfaces of the tool; e.g. a test which transcodes file A to file B and asserts correctness in some way. This kind of test should be absolutely achievable both in dev builds as well as when testing the deliverable artifact.