Irmin: Git-like distributed, branchable storage

In other words, I think I could implement what's being described as a very thin shim on top of git-annex. You'd just need a special git merge driver (the same as Irmin, which requires 3-way merge providers), but with the extra caveat that all three components of the merge have to be present in the local annex before a merge can take place.

This is just based on the article, and assumes I've understood correctly.

Is this accurate?

There are some interesting tricks going on from the native Irmin representation to the Git conversion (which is slightly less descriptive than Irmin and so virtual nodes are constructed to represent the extra data in Git). Will write that up in more detail in a future post I think, but for now:

https://github.com/mirage/irmin/blob/master/lib/backend/irmi... https://github.com/mirage/irmin/blob/master/lib/backend/irmi...

(is the Git serializer, and you can see in the interface how you can spawn an on-disk and in-memory Git).

and the simpler in-memory backend:

https://github.com/mirage/irmin/blob/master/lib/backend/irmi... https://github.com/mirage/irmin/blob/master/lib/backend/irmi...

where the implementation is mostly a noop since no mappings between representations needs to take place.

(Edited to note:) The reason for wanting an in-memory backend in the first place is that this is also very useful for IPC coordination. You could build a session layer where all the messages that go back-and-forth between two processes are recorded into an in-memory layer, and then when the whole process is done, the entire graph of communication can be dumped out to a Git tree as the log (for later analytics or debugging). If disk space is an issue, the Git tree can later be rebased to eliminate the intermediate communication commits. This is very, very useful for debugging.

In this scenario using this kind of verbose state logging starts to sound like a huge win, especially if tools exist to visualize how state mutation is different on various legs and use that information to infer dependency guesses between execution branches.

Perhaps the idea is more like a database, and some administration is manual. So, merges are like mini-migrations. Or, merges could be thrown back to a human user of the app to combine them (perhaps with some domain specific interface) - again, manual.