Live data from Hacker News

DegDB, an open-source distributed graph database

github.com

11–16 of 16 posts

Re: DegDB, an open-source distributed graph database

#11

Interesting proposition, but the majority of its design is flawed. Responding to the design doc: Truples are common in graph databases, however both gun and Neo4J use more property graphs (while Neo4j has mandatory edge nodes, gun does not), which in my personal opinion is actually useful while triples are more of an academic thing (note here, I am biased because I am the author of gun). He chucks conflict resolution…

Yeah, a lot of what you're saying I agree with.

In the several months since I wrote that I've changed my mind on a lot of things. The new design uses a sharded architecture (on hash of topic id) with nodes having a specific keyspace. This makes it much more robust and allows for actual consistency. Since all data will be treated equally, there will no longer be a penalty for less used data.

The main reason I originally thought that less used data should be penalized, is that it takes up a lot of resources for things that aren't used by the majority of users. However, that's a hard thing to track and makes it difficult to propagate inserts.

As for triples vs property graphs, they're functionally equivalent. I'm using triples because they've been shown to work quite well at scale such as Google's massive Knowledge Graph.

Re: DegDB, an open-source distributed graph database

#12
post #3

Nice. I've implemented basically the same thing about 5 years ago: https://code.google.com/p/graphpack/ (future dev on https://github.com/amitport/graphpack ) never had time to publish proper documentation though. you should also check out the paper that was published a few years after: http://onlinelibrary.wiley.com/doi/10.1002/spe.2226/abstract

That's really neat. Any reason you decided to abandon work on it?

Non pay-walled version: http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-get.cgi/2...

Re: DegDB, an open-source distributed graph database

#13
post #6

Nice to see. Is SPARQL support planned? I am wondering because it has a triplestore directory. In the meantime a more interesting production ready open source distributed graph database is worth looking at: https://www.blazegraph.com/ . It scales really well and will soon have GPU support for graph traversals. It has tinkerpop and SPARQL support.

https://github.com/google/badwolf this is another similar project being developed at Google. Badwolf has a SPARQL-like query language too.

BadWolf is interesting in its temporal aspect. But IMHO has dropped a bit to much from the RDF world to really take off.

Also I paid the price in early adoption of SPARQL/RDF. Not looking to repeat that with an even earlier adoption of a non standard system. Especially, if the temporal aspect does not appear in the data I work with.

Re: DegDB, an open-source distributed graph database

#14
post #11

Interesting proposition, but the majority of its design is flawed. Responding to the design doc: Truples are common in graph databases, however both gun and Neo4J use more property graphs (while Neo4j has mandatory edge nodes, gun does not), which in my personal opinion is actually useful while triples are more of an academic thing (note here, I am biased because I am the author of gun). He chucks conflict resolution…

Yeah, a lot of what you're saying I agree with. In the several months since I wrote that I've changed my mind on a lot of things. The new design uses a sharded architecture (on hash of topic id) with nodes having a specific keyspace. This makes it much more robust and allows for actual consistency. Since all data will be treated equally, there will no longer be a penalty for less used data. The main reason I original…

In the long having a triple based API does not mean you need to have a single triple table based storage.

In the SPARQL world that is actually quite interesting as different systems have very different data layouts while maintaining the same basic query language.

Comparing to the top post. I think triple systems are much more scalable than Neo4J even if not as popular. There are a few triple systems with a trillion node benchmarks. Even more with a 100 billion plus. Neo4J has at most ~34 billion relationships, and no more than 274 billion triples. Those are hard limits per current Neo4J documentation. But I have not heard of any Neo4j systems in production at that scale. While I know of at least one SPARQL system that is running with 4 trillion edges (http://allegrograph.blogspot.ch/2015/11/allegrograph-news-no...).

Re: DegDB, an open-source distributed graph database

#15
post #11

Interesting proposition, but the majority of its design is flawed. Responding to the design doc: Truples are common in graph databases, however both gun and Neo4J use more property graphs (while Neo4j has mandatory edge nodes, gun does not), which in my personal opinion is actually useful while triples are more of an academic thing (note here, I am biased because I am the author of gun). He chucks conflict resolution…

Yeah, a lot of what you're saying I agree with. In the several months since I wrote that I've changed my mind on a lot of things. The new design uses a sharded architecture (on hash of topic id) with nodes having a specific keyspace. This makes it much more robust and allows for actual consistency. Since all data will be treated equally, there will no longer be a penalty for less used data. The main reason I original…

Great thoughts. I was really impressed with how sharp your thinking was in your original document, just disagreed with its direction. Interesting to hear that you've thus revised things. Want to do a skype or something on this stuff? Shoot me an email mark@gunDB.io .

Re: DegDB, an open-source distributed graph database

#16
post #12
post #3

Nice. I've implemented basically the same thing about 5 years ago: https://code.google.com/p/graphpack/ (future dev on https://github.com/amitport/graphpack ) never had time to publish proper documentation though. you should also check out the paper that was published a few years after: http://onlinelibrary.wiley.com/doi/10.1002/spe.2226/abstract

That's really neat. Any reason you decided to abandon work on it? Non pay-walled version: http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-get.cgi/2...

As @barakm said in the comment above, "it's not something you just build without working with a couple people"...

It requires a lot of work to get to a "real" product which includes documentation, basic website, traction with users, etc,. I eventually got side-tracked with life and other work. I will be very happy to continue work on it if someone will be willing to actively help out.

Post reply on HN