Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irmin: Git-like distributed DB

mirage.io

Post reply on HN
Print view View original

21–26 of 26 posts

Re: Irmin: Git-like distributed DB

#21

Post by avsm »

amelius wrote: post #19

Nice, but there is a fundamental problem with the three-way merge: the guarantees about the result are very weak, and it may require special attention to resolve merge conflicts.

That's sort of the entire point. The idea is that you build datastructures using this library that operate over the three-way merge, and provide their own guarantees about merge conflicts.

For instance, a weakly consistent data structure could promise never to raise a merge conflict, and therefore be safe to compose. A stronger one could raise more precise merge exceptions depending on the exact error, which ripple up to the application.

An example of an application handling this sort of merge error is the Cuekeeper TODO manager (which is a pure JavaScript Irmin app that uses HTML5 Localstorage/IndexedDB). Try opening http://test.roscidus.com/CueKeeper/ in two tabs and creating conflicting changes, and see the Irmin merge error ripple up to the UI.

Re: Irmin: Git-like distributed DB

#22

Post by amelius »

mbrock wrote: post #20
amelius wrote: post #19

Nice, but there is a fundamental problem with the three-way merge: the guarantees about the result are very weak, and it may require special attention to resolve merge conflicts.

Can you expand on the weak guarantees? Sure, conflicts need to be handled. That's a fundamental problem with reality!

Well, the problem with a merge is that you lose information about the "intention" of the operation that resulted in those specific versions to be merged. All you've got is the end result of those operations. This becomes especially troublesome if there are invariants that need to hold over multiple data-items that have changed in a 3-way merge.

Re: Irmin: Git-like distributed DB

#23

Post by david_ar »

Natanael_L wrote: post #15
csense wrote: post #10

How does this compare / contrast with IPFS?

IPFS isn't designed as a database, more as a lookup layer for files

You'll soon be able to store arbitrary data structures on IPFS [1], and we're discussing how to perform merging in a decentralised manner [2]. I'm actually quite interested in having a persistent data storage system (like Irmin) backed by IPFS in the future.

[1] https://github.com/ipfs/go-ipld [2] https://github.com/ipfs/notes/issues/40

Re: Irmin: Git-like distributed DB

#24

Post by mbrock »

amelius wrote: post #22
mbrock wrote: post #20

Earlier quoted context omitted.

Can you expand on the weak guarantees? Sure, conflicts need to be handled. That's a fundamental problem with reality!

Well, the problem with a merge is that you lose information about the "intention" of the operation that resulted in those specific versions to be merged. All you've got is the end result of those operations. This becomes especially troublesome if there are invariants that need to hold over multiple data-items that have changed in a 3-way merge.

I would probably understand better with a concrete example. It's not clear what "intention" means, and why it can't be encoded as part of whatever objects are being merged.

Re: Irmin: Git-like distributed DB

#25

Post by talex5 »

avsm wrote: post #21
amelius wrote: post #19

Nice, but there is a fundamental problem with the three-way merge: the guarantees about the result are very weak, and it may require special attention to resolve merge conflicts.

That's sort of the entire point. The idea is that you build datastructures using this library that operate over the three-way merge, and provide their own guarantees about merge conflicts. For instance, a weakly consistent data structure could promise never to raise a merge conflict, and therefore be safe to compose. A stronger one could raise more precise merge exceptions depending on the exact error, which ripple u…

Note: CueKeeper is a bit unusual here. Merges always succeed, but it adds a note to the item saying what it did to resolve the conflict.

e.g. if you rename an action "orig" to "a" and to "b" and then merge, you'll end up with an action called "a" with a note saying the change to "b" was discarded.

BTW: it's easier to generate merge conflicts using this web interface, which lets you run two instances in one window:

http://roscidus.com/blog/blog/2015/04/28/cuekeeper-gitting-t...

Re: Irmin: Git-like distributed DB

#26

Post by talex5 »

marknadal wrote: post #5

From your description I'm not quite sure where in the stack Irmin belongs? Is this to be used by web application developers? I assume note, as this looks like it is targeting more OS level development work? Pretty cool stuff. I am also working on a distributed database, https://github.com/amark/gun , that operates at the high level (web/javascript) rather than the low level. Although it looks like Irmin can be used i…

Like most OCaml libraries, Irmin can be used in the browser by compiling your program with js_of_ocaml.

My `irmin-js` experiment provides a Javascript API to Irmin, so you can write your application in Javascript too, rather than OCaml. My JS isn't very good though; here's what my example code ended up looking like:

https://github.com/talex5/irmin-js/blob/master/examples/test...

Post reply on HN