Jepsen: Dgraph 1.1.1

I find Dgraph to be one of the most interesting of the current batch of non-relational data stores.

I've wanted a robust, easy-to-use graph database for years (ever since reading about the crazy brilliant graph database stuff that goes on inside Facebook https://www.facebook.com/notes/facebook-engineering/tao-the-... ) and Neo4J never really cut it for me.

Watching Dgraph mature - and survive two rounds of Jepsen with relatively decent marks - is intriguing. I don't have a project that needs it yet but maybe something will come up soon.

Neo4j is... conditionally alright. The main problem is that their marketing has a bad case of MongoDB syndrome.

N4J: "We sell cars. Our cars can be anything you want, and are at least pretty good at everything, and often great and even better than any other possible car you can get!"

Client: "Cool, I need a spacious four-door sedan with good gas mileage that's also a race car"

N4J: "Oh yeah, definitely, no problem, this is the product for you, totally fits your use case!"

Client orders car, it arrives, is in fact a delivery van that likes to stall at traffic lights

Client: "Uh. This would be useful if I needed a delivery van for a route with no stops, I suppose."

one thing i'm not a huge fan of is dgraph's UID model, which is effectively an auto-incrementing uint across the entire cluster. because it auto increments server side, it's non-deterministic; ingesting 10 nodes before 10 others means that the UIDs will change across despite being the same XID. there is a way to use "blank nodes" to link nodes and edges with non-int UIDs, but that is only per-mutation, not per-commit or per-transaction. there is no way to tell dgraph what the UID should be.

that means that if you have externally unique IDs that you have infrastructure around, you are either caching that node's UID externally or doing an XID->UID lookup in order to create edges.

there is a bulk loader but that's only available in HA mode, and the UID:XID map it generates is obviously for data you already had in flat files (or whatever). so it's ok for static data sets, but not ideal for live updating data.

the gRPC API also has strange undocumented (AFAICT) behavior where even smallish batches of 100 hit some unspecified gRPC limit, so you need smaller batches ergo more commits ergo more wasted compute.

I think the other way around. Dgraph was thought to be distributed. It cannot be "atomically" distributed based on entities. But in smaller parts of an entity. If each instance of Dgraph takes care of a small part of that entity, it becomes virtually more performatic.

Imagine, in an abstract way, that it is a hashing process. Where the attributes are spread over several instances in the cluster and Dgraph just "decodes" it for you quickly because it already has the key.

Could it be based on entities? yes, but I don't think it would bring any benefit. Imagine that you have 10 instances, and you have 10 entities. You have 3 entities with many edges (values and outgoing edges) and 8 "light" entities, without much information. Well, you will leave 3 entities in high usesage of resources while 8 entities are idle because they do not have much information to deliver to all query and mutation requests in the Cluster.

A system based on attributes in a distributed way is much more performance tho.

I'd similarly been evaluating a Python client implementation a while back and found the developer experience a little rough around the edges[1].

It's reassuring to see Dgraph undergoing the full Jepsen treatment, even if it highlights that there's still a bit of work to do, and further stability to prove.

[1] - https://github.com/dgraph-io/pydgraph/issues/94