Lots of progress for Debian's reproducible builds
1–10 of 35 posts
Re: Lots of progress for Debian's reproducible builds
#2The reproducible builds talk at 31C3 also does a nice job of explaining some of the many possible attack vectors that make reproducible builds desirable, and many of the subtleties involved in making it work:
http://media.ccc.de/browse/congress/2014/31c3_-_6240_-_en_-_...
Re: Lots of progress for Debian's reproducible builds
#3Baserock (http://wiki.baserock.org) may have a repeatable build of OpenEmbedded for automotive systems.
Re: Lots of progress for Debian's reproducible builds
#4It can be surprisingly difficult. Funnily enough moving from svn git in one project I know of probably did a lot of the necessary work to achieve this, by having to remove reliance on $SVN tags and pre/post-"build commits" which used to be a part of the release process.
It's an interesting use-case for Docker as well: you can ship the build environment (or its Dockerfile describing it) for people to run builds under the same env as the official released build.
Re: Lots of progress for Debian's reproducible builds
#5I actually did some work making debootstrap reproducible. So even if the 100 or so .deb builds it depends on are reproducible, then the chroot image resulting from debootstrap will not be reproducible byte-for-byte, due to the debootstrap shell script itself and the tools it calls.
Offhand, I remember that /etc/{passwd,group} are copied from the host machine by design. There is also a random seed file, to save entropy across reboots. And there is some nondeterminism in the dynamic linker cache AFAIK. And timestamps in logs.
If anyone is interested in this let me know.
Re: Lots of progress for Debian's reproducible builds
#6Debian does amazing amounts of system-wide initiatives. Off the top of my head, there are multiarch https://wiki.debian.org/Multiarch, clang rebuild http://clang.debian.net/, and automated code analysis https://qa.debian.org/daca/.
Re: Lots of progress for Debian's reproducible builds
#7It can be surprisingly difficult. Funnily enough moving from svn git in one project I know of probably did a lot of the necessary work to achieve this, by having to remove reliance on $SVN tags and pre/post-"build commits" which used to be a part of the release process. It's an interesting use-case for Docker as well: you can ship the build environment (or its Dockerfile describing it) for people to run builds under…
I had the same hope for Docker, that it would ease reproducible builds. It quickly became clear to me how much work still needs to be done for that to be realistic. I maintain a packaging of Meteor using Docker designed to increase reproducibility (https://registry.hub.docker.com/u/danieldent/meteor/). It does some checksums on the Meteor code, but the build process itself introduces entropy, and Dockerfile doesn't really have the primitives to make it easy to prevent that.
Many many build processes have hidden from-the-network dependencies which are yet another huge source of problems (both in terms of having high-availability for the build process and in terms of understanding exactly what is getting built).
All of the upstream Docker images (including the bottom base images which most people never look at carefully) would need to be built in a reproducible way. And the Docker code would need to actually verify checksums more carefully than it currently does.
Having projects like Debian doing work like this will make it a lot easier for everything else to become reproducible. Which in addition to the security benefits is also pretty useful when bug hunting - having a system that makes it clear which (if any) of your dependencies changed narrows the list of things which need to be checked when something breaks. Ubuntu Core, Snappy, and Nix are also on my list of things to watch in this area.
Re: Lots of progress for Debian's reproducible builds
#8Little bit of related trivia : Lunar (J.Bobbio) worked on hOp, a GHC based Haskell micro kernel so you can write drivers in it. See https://github.com/dls/house. A knowledgeable fellow.
Re: Lots of progress for Debian's reproducible builds
#9Baserock ( http://wiki.baserock.org ) may have a repeatable build of OpenEmbedded for automotive systems.
What's the relationship between Baserock and OpenEmbedded? Glancing through OpenEmbedded's wiki page, it seems to a recipe/build system for embedded linux. Baserock seems to be in the same league.
Re: Lots of progress for Debian's reproducible builds
#10It can be surprisingly difficult. Funnily enough moving from svn git in one project I know of probably did a lot of the necessary work to achieve this, by having to remove reliance on $SVN tags and pre/post-"build commits" which used to be a part of the release process. It's an interesting use-case for Docker as well: you can ship the build environment (or its Dockerfile describing it) for people to run builds under…
I had the same hope for Docker, that it would ease reproducible builds. It quickly became clear to me how much work still needs to be done for that to be realistic. I maintain a packaging of Meteor using Docker designed to increase reproducibility ( https://registry.hub.docker.com/u/danieldent/meteor/ ). It does some checksums on the Meteor code, but the build process itself introduces entropy, and Dockerfile doesn't…
Ah, I should've thought twice about that - I'm not even attempting reproducible builds in my own work, but I've often seen my Dockerfiles have apt-get-induced shenanigans due to the particular day or half-broken mirror I happen to be running the build from.
NixOS is definitely on my list, although for different reasons (whilst I love using and babysitting Debian systems, orchestrating it is eating up time that could be better spent on something like NixOS).