Live data from Hacker News

OrbitDB: Peer-to-peer databases for the decentralized web

github.com

101–110 of 119 posts

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#101

Earlier quoted context omitted.

almost! :) let me elaborate on few points. > you have to reference all of the current concurrent root nodes of the data structure, in effect becoming the new single root node correct, and more precisely the union of heads is the current "single root node". in practise, and this is where the merge strategy comes in, the "latest value" is the value of the event that is "last" (as per LWW sorting). > and your event data…

> in a log database, the ordered log would be eg. 1 How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? > if 4 and 6 are both a set operation on key a, it would mean that key a would have the value from 6 and the next write to key a would overwrite the value in a. Yes, I mean for all of my events to be reads and writes of the same key. And you've proven my point, I think:…

> How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"?

I jumped the gun there and made an assumption that the value of a node is the LWW ordering :) Ok, so without that assumption, the DAG

  1 --> 2 --> 3 --> 4 
         `--> 5 --> 6
...are the values of the operations that the DAG represents, ie. values of a key, so we need to look at the Lamport clocks (or Merkle Clocks when the operations are hashed as a merkle dag) of each operation, represented here as ((ts, id), key, value):

  ((0, x), a, 1) --> ((1, x), a, 2) --> ((2, x), a, 3) --> ((3, x), a, 4)
                                   `--> ((2, y), a, 5) --> ((3, y), a, 6)

which one is the latest value for key a? Which updates, semantically, were lost? In a non-CRDT system, which value (4/x or 6/y) is or should be displayed and considered the latest?

> This is a problem! If you claim to be a CRDT and offline-first or whatever, then as a user, I expect that the operations I make while I'm disconnected aren't just going to be destroyed when I reconnect, because someone else happened to be using a computer with a lexicographically superior hostname (or however you derive your vector clocks).

You're conflating the data(base) model with the log and we can't generalize that all cases of data models or merge conflict are cases of "I expect all my operations to be the latest and visible to me". They are semantically different. If the writes are on the same key, one of them has to come first if the notion of "latest single value" is required. If the writes are not on the same key, or not key-based, multiple values appear where they need to. What we can generalize is that by giving a deterministic sorting function, the "latest value" is the same for all participants (readers) in the system. From data structure perspective this is correct: given same set of operations, you always get the same result. For many use cases, LWW works perfectly fine, and if your data model requires a "different interpretation" of the latest values, you can pass in your custom merge logic (=sorting function) in orbitdb. The cool thing is, that by giving a deterministic sorting function for a log, you can turn almost any data structure to a CRDT. How they translate to end-user data model will depend (eg. I wouldn't model, say, "comments on a blog post" as a key-value store).

If you're curious to understand more, I think the model is best described in the paper "OpSets: Sequential Specifications for Replicated Datatypes" [1]. Another two papers, from the same author, that may also help are "Online Event Processing" [2] and "Moving Elements in List CRDTs" [3] which show how by breaking down the data model to be more granular than "all or nothing", composing different CRDTs give arise to new CRDTS, which I find beautiful. Anything, really, that M. Kleppmann has written about the topic is worth a read :)

[1] https://arxiv.org/pdf/1805.04263.pdf [2] https://martin.kleppmann.com/papers/olep-cacm.pdf [3] https://martin.kleppmann.com/papers/list-move-papoc20.pdf

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#102

Earlier quoted context omitted.

> in a log database, the ordered log would be eg. 1 How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? > if 4 and 6 are both a set operation on key a, it would mean that key a would have the value from 6 and the next write to key a would overwrite the value in a. Yes, I mean for all of my events to be reads and writes of the same key. And you've proven my point, I think:…

> How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? I jumped the gun there and made an assumption that the value of a node is the LWW ordering :) Ok, so without that assumption, the DAG 1 --> 2 --> 3 --> 4 `--> 5 --> 6 ...are the values of the operations that the DAG represents, ie. values of a key, so we need to look at the Lamport clocks (or Merkle Clocks when the ope…

OK, I understand now. I guess my points then translate to:

1. Modeling an append-only log as a CRDT is trivial

2. Building a database on top of a "CRDT" append-only log doesn't make the database a CRDT

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#103

Earlier quoted context omitted.

> How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? I jumped the gun there and made an assumption that the value of a node is the LWW ordering :) Ok, so without that assumption, the DAG 1 --> 2 --> 3 --> 4 `--> 5 --> 6 ...are the values of the operations that the DAG represents, ie. values of a key, so we need to look at the Lamport clocks (or Merkle Clocks when the ope…

OK, I understand now. I guess my points then translate to: 1. Modeling an append-only log as a CRDT is trivial 2. Building a database on top of a "CRDT" append-only log doesn't make the database a CRDT

No, on both. See above.

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#104
post #90

Earlier quoted context omitted.

> I guess performance and robustness were not a priority. You'd be surprised how well JS does on both fronts, in addition to being able to run across platforms :)

Performance-wise, JS is not too bad when you have lots of IO going on, as in this case... but robustness? You gotta be kidding! JS is the poster child of a language does NOT have robustness as one of its attributes. Just about every single one of its design decisions makes robustness difficult.

You should check your definition.

Define robust:

1. Strong and healthy; vigorous

- (of a process, system, organization, etc.) able to withstand or overcome adverse conditions.

JS is the most robust language we have according to the main definition, however you measure it. What other language is as alive as JS right now? What other language has as much programmer attention?

JS is also easily robust according to the noted sub-definition. What other language would survive the web? The web, an environment that could easily be described as "very adverse"... What other language is ready to run in a browser where it will be dynamically and continuously mixed with modules from many different sources, without breaking? JS was built for this.

What other popular language is so easily runnable cross platform? What other popular language lets you program with both OOP and functional paradigms while also being as popular as JS? What other popular language lets you monkey patch things to fit a piece of code into any situtation?

Robust means all of these things to me. What does robust mean to you and what is your example of the most robust programming language? You didn't say...

None of these fit the bill: Python, Ruby, C, C++, Golang, Rust, C#.

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#105

Earlier quoted context omitted.

Well, I imagine you'll be happy to know that some of us have begun IPFS implementation in Rust: https://github.com/ipfs-rust/rust-ipfs

Definitely a much better choice

Can you elaborate why you think Rust is a better choice?

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#106

Earlier quoted context omitted.

> in a log database, the ordered log would be eg. 1 How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? > if 4 and 6 are both a set operation on key a, it would mean that key a would have the value from 6 and the next write to key a would overwrite the value in a. Yes, I mean for all of my events to be reads and writes of the same key. And you've proven my point, I think:…

> How do you know that? It's not inferrable from the DAG. Is sequencing also provided "a layer up"? I jumped the gun there and made an assumption that the value of a node is the LWW ordering :) Ok, so without that assumption, the DAG 1 --> 2 --> 3 --> 4 `--> 5 --> 6 ...are the values of the operations that the DAG represents, ie. values of a key, so we need to look at the Lamport clocks (or Merkle Clocks when the ope…

[deleted]

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#107
post #19

OrbitDB is one of the key dependencies in 3box, an awesome tool for building decentralized apps where the user controls their own data. https://3box.io/

3box makes OrbitDB really easy to have a persistent easy to use!

If you try to go with just OrbitDb you will have to handle manage the ipfs nodes and pin data so it stays persistent and loads faster but also handle the permissions too.

But with 3Box makes it easier to do all of the permissions and loading all data faster + always persistent and/or public or private data, and you know what's amazing? the way you can store data? for example if you want to build a chat app you can use Messaging (Threads) to make quickly a chat app! https://docs.3box.io/api/messaging

Enjoy!

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#108
post #9

Earlier quoted context omitted.

One of the main difference is it is based on IPFS

What does GUN use instead of IPFS? What are the pros and cons of the 2 approaches?

I'm not sure about GUN But one obvious pro of IPFS is you leverage the huge network of IPFS nodes.

Re: OrbitDB: Peer-to-peer databases for the decentralized web

#110
post #6

What is the use case for this ?

The use case is shared, mutable data structures that don't rely on central coordination or control.

Does that even make sense? Why wouldn't you want centralized coordination of your data structures? We could go back to people emailing emailing Excels to each other is that's what you want.
Post reply on HN