Anyone tried to use this with Spring?
Titan: A Highly Scalable, Distributed Graph Database
31–40 of 41 posts
Re: Titan: A Highly Scalable, Distributed Graph Database
#32"Vertex-centric indices provide vertex-level querying to alleviate issues with the infamous super node problem"
Does that mean, for each vertex, is it's sub-graph indexed?!
Re: Titan: A Highly Scalable, Distributed Graph Database
#33Earlier quoted context omitted.
You mean CAP theorem? http://en.wikipedia.org/wiki/CAP_theorem I imagine it is either the A or the P that gets to be the victim, but I'm not sure which in this case.
No, I am referring to the set of problems related to graph partitioning. This is essentially the same underlying problem that is the source of why distributed NoSQL databases do not support join operations. In the case of NoSQL databases, they simply do not support joins because it is not a core operations. (Technically you can still do a join, it just has terrible scaling characteristics.) The fundamental operation…
I've been working on the graph partitioning issues for distirbuted graphdbs for a while (my own pet project that my brain won't let me give up on) but it's been from the perspective of someone who's not been keeping up to date with the Comp Sci literature. Have you got any suggestions for canonical papers from the last decade or so, for the state of the art?
Re: Titan: A Highly Scalable, Distributed Graph Database
#34Also, What is interesting about a graph database relative to simple key-value database? Storing edges of a graph is trivial for a key-value store and so it seems like any key-value store could let store the basic graph structure? Do graph databases support graphic-specific queries and indices?
Read Neo4j's documentation [1](well, don't read; just gloss over it). You'll be blown away by how much it can do. I recommend looking at chapter 5. An example, with the actual query:
-----------
5.3.1. Co-Favorited Places - Users Who Like x Also Like y
Find places that people also like who favorite this place:
* Determine who has favorited place x.
* What else have they favorited that is not place x.
Query:
START place=node:node_auto_index(name = "CoffeeShop1")
MATCH placestuff
RETURN stuff.name, count(*)
ORDER BY count(*) DESC, stuff.name
[1]: PDF: http://docs.neo4j.org/pdf/neo4j-manual-milestone.pdf , online: http://docs.neo4j.org/chunked/milestone/
Re: Titan: A Highly Scalable, Distributed Graph Database
#35Earlier quoted context omitted.
You could have said the same thing about generalized distributed computing as well. Much like with MapReduce, the trick is to recognize that there is an 80% solution that works quite well (based around BSP): http://kowshik.github.com/JPregel/pregel_paper.pdf http://googleresearch.blogspot.com/2009/06/large-scale-graph... The trick is not to build a generalized graph operator solution. It's to have a specialized graph…
Approaches like Pregel have been the canonical way of dealing with large graph problems for many years. Unfortunately, most interesting graph analytic problems do not fit into that model because the "graph-like" aspect is still limited to problem sizes that fit conventional algorithms. For example, if you can reduce a trillion-edge graph analysis problem into a billion-edge graph plus some other stuff (usually materi…
> Unfortunately, most interesting graph analytic problems do not fit into that model because the "graph-like" aspect is still limited to problem sizes that fit conventional algorithms.
"interesting" != "useful"
Again, the same thing has been true with distributed computing in general. MapReduce & Hadoop are pretty much the antithesis of where distributed computing research for the last ~20 years had been working, because MapReduce solves what is nearly an "embarrassingly parallel" problem.
> There are many, many graph analytic problems that require a true graph that is orders of magnitude larger than what can be partitioned even after accounting for graph reduction techniques such as those used in Pregel.
There are even more distributed computing algorithms that don't fit in to MapReduce terribly well (and really, it's not that algorithms don't work with MapReduce/Pregel, it's that they don't work well), but it is still quite useful.
Turns out, the reason it's the Holy Grail is that it is just flat out hard to do (provably so). While what Titan/Pregel do isn't nearly is difficult, it is surprisingly difficult to do at massive scale, so just doing the simple stuff they do is quite useful and game changing.
Re: Titan: A Highly Scalable, Distributed Graph Database
#36Also, What is interesting about a graph database relative to simple key-value database? Storing edges of a graph is trivial for a key-value store and so it seems like any key-value store could let store the basic graph structure? Do graph databases support graphic-specific queries and indices?
Storing graphs in a key/value store is trivial. Querying them is not. Gremlin is much more powerful than anything you can home bake.
Re: Titan: A Highly Scalable, Distributed Graph Database
#37We are going to address alot of these questions in our presentation tonight. However, for the Hacker News crew that won't be there tonight, here is an early release of the Titan talk. https://speakerdeck.com/u/okram/p/titan-the-rise-of-big-grap... http://titanbiggraphdata.eventbrite.com/ Enjoy!, Marko.
Great presentation.
Re: Titan: A Highly Scalable, Distributed Graph Database
#38Earlier quoted context omitted.
No, I am referring to the set of problems related to graph partitioning. This is essentially the same underlying problem that is the source of why distributed NoSQL databases do not support join operations. In the case of NoSQL databases, they simply do not support joins because it is not a core operations. (Technically you can still do a join, it just has terrible scaling characteristics.) The fundamental operation…
I've been working on the graph partitioning issues for distirbuted graphdbs for a while (my own pet project that my brain won't let me give up on) but it's been from the perspective of someone who's not been keeping up to date with the Comp Sci literature. Have you got any suggestions for canonical papers from the last decade or so, for the state of the art?
There has not been much that is both new and interesting in the graph partitioning literature in a long time. What you already know is probably not too far off from what is in the current literature. The literature on this topic has been stagnant for years.
Solutions to the graph partitioning problem exist and among people doing high-end graph analytics this has been rumored for years now. It just is not published and people that know how it is done are slathered in NDAs. I know of two different (related) algorithms for parallelizing graph analysis. IBM Research currently has the most advanced algorithms for graph analysis and they disclose very little about how they work.
Re: Titan: A Highly Scalable, Distributed Graph Database
#39We are going to address alot of these questions in our presentation tonight. However, for the Hacker News crew that won't be there tonight, here is an early release of the Titan talk. https://speakerdeck.com/u/okram/p/titan-the-rise-of-big-grap... http://titanbiggraphdata.eventbrite.com/ Enjoy!, Marko.
Great presentation.
Also it appears like a great way to take speakerdeck down...
Re: Titan: A Highly Scalable, Distributed Graph Database
#40Anyone tried to use this with Spring?
Why the downvote? Spring Data Neo4j [1] works excellent, it's pretty popular and I'm interested has anyone tried working with TITAN in a Spring project(probably using the Blueprints API).