Live data from Hacker News

Three Pillars of Reproducible Builds

fossa.com

11–15 of 15 posts

Re: Three Pillars of Reproducible Builds

#11
post #7

These three aren't enough, you also need to take care of not storing build timestamps, hostnames, timezones, sorting and more: https://reproducible-builds.org/docs/

Some of that is mentioned, e.g.

> Build steps that use system time to generate timestamps.

> Builds that change behavior based on currently set environment variables but don’t commit environment variable configurations.

Re: Three Pillars of Reproducible Builds

#12
post #2

On the JVM, maven doesn’t make this particularly easy. It’s possible to try to store dependencies locally instead of shared in a global m2 repository, but it’s difficult to stop maven from adding the current time in jars or wars… It’s as if all the default settings are the opposite of what they should be for reproducible builds. Any idea if there is a project to try to improve things with maven or with another JVM to…

> Any idea if there is a project to try to improve things with maven or with another JVM tool? (Grade, sbt, etc.)

We've found SBT to be less reproducible than Maven. In particular, its "configuration file" (build.sbt) is actually executable Scala code (and highly imperative too, e.g. appending to mutable dependency lists). I've seen projects which choose different dependencies based on env var settings, string matches, etc.

I've also seen projects which add pre/post steps to a test suite, for spinning-up and tearing-down a mock database (the dynamodb-local SBT plugin). The crazy part about that, is that SBT only becomes aware of the plugin when it's about to execute the test suite; hence it doesn't appear in any dependency lists, so we can't automatically fetch it ahead-of-time. By the way, that plugin itself works by downloading and running a "latest.zip" file from an AWS URL....

Re: Three Pillars of Reproducible Builds

#13

One of the most fun non-determinism bugs I have worked on was the result of using an associative container with the key type being a pointer (like a std::map or similar), and then iterating over this container. Since the order and value of dynamically allocated pointers is non-deterministic, this resulted in diverging behaviour at some point. Better be sure that all your tools used during the build don't do this kind…

With ASLR off, the order and value should be identical between runs on the same malloc implementation, as stochastic allocators are not in common use

Re: Three Pillars of Reproducible Builds

#14
post #13

One of the most fun non-determinism bugs I have worked on was the result of using an associative container with the key type being a pointer (like a std::map or similar), and then iterating over this container. Since the order and value of dynamically allocated pointers is non-deterministic, this resulted in diverging behaviour at some point. Better be sure that all your tools used during the build don't do this kind…

With ASLR off, the order and value should be identical between runs on the same malloc implementation, as stochastic allocators are not in common use

Not when multi-threading is involved, I would think. That or timing-dependent code making allocations.

Re: Three Pillars of Reproducible Builds

#15
post #13

Earlier quoted context omitted.

With ASLR off, the order and value should be identical between runs on the same malloc implementation, as stochastic allocators are not in common use

Not when multi-threading is involved, I would think. That or timing-dependent code making allocations.

That's true.
Post reply on HN