Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reproducible C++ builds by logging Git hashes

jgarby.uk

Post reply on HN
Print view View original

1–10 of 45 posts

Re: Reproducible C++ builds by logging Git hashes

#2

Post by j4cobgarby »

Here's a short writeup of a bit of my build system for a project I'm working on. It's pretty simple, and is just a relatively clean way of recording the repository state when code was compiled, so I can reproduce results later on. Just thought the interaction between git, cmake, and C++ was a bit nice!

Re: Reproducible C++ builds by logging Git hashes

#4

Post by danudey »

A simpler way to do this, especially if you do tagging in your repositories, is to use `git describe`. For example:

    $ git describe --dirty
    v1.4.1-1-gde18fe90-dirty
The format is --g-.

If the repo isn't dirty, then the hash you get excludes that part:

    $ git describe --dirty
    v1.4.1-1-gde18fe90
If you're using lightweight tags (the default) and not annotated tags (with messages and signatures and etc) you may want to add `--tags` because otherwise it'll skip over any lightweight tags.

The other nice thing about this is that, if the repo is not -dirty, you can use the output from `git describe` in other git commands to reference that commit:

    $ git show -s v1.4.1-1-gde18fe90
    commit de18fe907edda2f2854e9813fcfbda9df902d8f1 (HEAD -> 1.4.1-release, origin/HEAD, origin/1.4.1-release)
    Author: rockowitz 
    Date:   Sun May 28 17:09:46 2023 -0400

        Create codacy.yml

Re: Reproducible C++ builds by logging Git hashes

#5

Post by ikety »

chuckadams wrote: post #3

Give Nix a look sometime, it takes this to a whole new level by including all of the build dependencies in the hash, and their build dependencies and so on. The standard flake workflow even includes the warning about having uncommitted files.

It's quite odd to me that Nix or something similar like Mise isn't completely ubiquitous in software. I feel like I went from having issues with build dependencies to having that aspect of software development completely solved as soon as I adopted Nix.

I absolutely can't imagine not using some kind of tool like this. Feels as vital as VCS to me now.

Re: Reproducible C++ builds by logging Git hashes

#6

Post by chuckadams »

ikety wrote: post #5
chuckadams wrote: post #3

Give Nix a look sometime, it takes this to a whole new level by including all of the build dependencies in the hash, and their build dependencies and so on. The standard flake workflow even includes the warning about having uncommitted files.

It's quite odd to me that Nix or something similar like Mise isn't completely ubiquitous in software. I feel like I went from having issues with build dependencies to having that aspect of software development completely solved as soon as I adopted Nix. I absolutely can't imagine not using some kind of tool like this. Feels as vital as VCS to me now.

We'd have been a lot further along if tools like make had ever adopted hashes for freshness checking rather than timestamps. We'd have ccache built in to make, make could hash entire targets, and now we're halfway to derivations. Of course that's handwaving over the tricky problem of making sure targets build reproducibly, but perhaps compiler toolchains would have taken more care to ensure it.

Re: Reproducible C++ builds by logging Git hashes

#8

Post by groby_b »

This is many useful things, but it's far from a reproducible C++ build. That'd require you ensure bit-for-bit identic builds when you reproduce, and logging the repository state is just a tiny first step to get there.

https://nikhilism.com/post/2020/windows-deterministic-builds... is a good resource on some of the other steps needed. It's... a non-trivial journey :)

Re: Reproducible C++ builds by logging Git hashes

#10

Post by WorldMaker »

danudey wrote: post #4

A simpler way to do this, especially if you do tagging in your repositories, is to use `git describe`. For example: $ git describe --dirty v1.4.1-1-gde18fe90-dirty The format is - -g - . If the repo isn't dirty, then the hash you get excludes that part: $ git describe --dirty v1.4.1-1-gde18fe90 If you're using lightweight tags (the default) and not annotated tags (with messages and signatures and etc) you may want to…

`git describe` is great.

Also, if you don't feel ready to commit to tagging your repository you can start with the `--always` flag which falls back to just the short commit hash.

The article's script isn't far from `git describe --always --dirty`, which can be a good place to start, and then it gets better as you start tagging.

Post reply on HN