Live data from Hacker News

Graph Databases 101

cray.com

61–70 of 107 posts

Re: Graph Databases 101

#61
post #49
post #15

Earlier quoted context omitted.

But if you want to scale to more than two instances you will have to pay a $50k license. $50.000,00 dollars. Which is far too expensive for startups.

Look, I only wrecked a semester of research by chasing the slippery promises of graph databases. Don't wreck your entire startup on them.

Explain

Re: Graph Databases 101

#62
post #29

It introduces false dichotomy "graph vs relational". In fact, most (if not all) graph algorithms can be expressed using linear algebra (with specific addition and multiplication). And matrix multiplication is a select from two matrices, related with "where i=j" and aggregation over identical result coordinates. The selection of multiplication and addition operations can account for different "data stored in links and…

linear algebra != relational calculus

Your argument is effectively because Haskell can be implemented in C, there is a false dichotomy between the two languages.

Re: Graph Databases 101

#63
post #49

Earlier quoted context omitted.

Look, I only wrecked a semester of research by chasing the slippery promises of graph databases. Don't wreck your entire startup on them.

Explain

ConceptNet [1] started out as an academic project that I was responsible for for a while. Since then I've left to start a company but I still maintain ConceptNet and build lots of stuff on it.

[1] http://conceptnet5.media.mit.edu

Here's a list of databases, some of them graph databases, some of them barely databases, where I've tried to store and look up edges of ConceptNet:

  - SQLite
  - PostgreSQL
  - MongoDB
  - Some awful IBM quad-store
  - HypergraphDB
  - Tinkerpop
  - Neo4J
  - Solr
  - Riak
  - SQLite with APSW to speed up importing
  - Just a hand-rolled hashtable on disk
Here are the systems that have succeeded to any extent, in that I could do simple things with them and they didn't collapse:

  - PostgreSQL
  - SQLite with APSW to speed up importing
  - Just a hand-rolled hashtable on disk
The time when I tried Tinkerpop, HypergraphDB, and Neo4J because I had a graph and graph databases are supposed to be good at graphs was particularly terrible. Graph databases seem to only be good at dealing with graphs so small that anything can deal with them.

If this has changed, please point me at an open-source graph database that's not terrified of gigabytes. (No trying to sell me SaaS, please.)

Re: Graph Databases 101

#64
post #15
post #5

Earlier quoted context omitted.

Depends on what kind of data and graph you are going to store/use. Neo4j is quite popular, cypher isn't very hard to learn, and it has lots of examples. Might be a good choice for a beginner. https://en.wikipedia.org/wiki/Graph_database#List_of_graph_d...

But if you want to scale to more than two instances you will have to pay a $50k license. $50.000,00 dollars. Which is far too expensive for startups.

Not entirely accurate. They have a startup program. Neo4j Enterprise is available for free for startups up to 20 employees.

Source: http://neo4j.com/startup-program/

Re: Graph Databases 101

#65
post #48
post #36

Earlier quoted context omitted.

Cray is a brand name that has been passed around between half a dozen companies (including Sun and SGI) dotted by various kinds of product reboots and commercial failures. Cool stuff but supercomputing isn't the most financially sound business it seems. The current name holder is the company previously called Tera, originally famous for making an aggressively multithreaded HPC computer.

I'm not clear how Sun ever owned Cray care to explain? The provenance was Cray Research -> SGI -> Tera/Cray according to those that have been around since the Cray Research days. Source: err, I work here and asked a couple people a few cubes over. :) The Sun deal was apparently more SGI wouldn't be caught dead with a supercomputer that ran on sparc so it got sold off to Sun.

My bad about Sun. Yep, Sun just bought the SPARC-based stuff & developed further under their own brand (E10k etc).

Re: Graph Databases 101

#66
post #6
post #2

Question as someone new to graph databases: Are there any open source graph databases worth looking into?

Cayley is a good option; we use it in production. https://github.com/google/cayley

Or if you like the idea of using a graph database developed by a secretive organisation full of geniuses dedicated to collecting and organising the world's information, but would rather it was public-sector, there's Gaffer:

https://github.com/GovernmentCommunicationsHeadquarters/Gaff...

Re: Graph Databases 101

#68

I have spent a lot of time figuring out how to deal with a large graph a couple of years ago. My conclusion - there will never be such a thing as a "graph database". There are many efforts in this area, someone here already mentioned SPARQL and RDF, you can google for "triple stores", etc. There are also large-scale graph processing tools on top of Hadoop such as Giraph or Graphx for Spark. For the particular project…

Years ago PostgreSQL already support recursive query, and in Oracle you have CONNECT BY. I have only used the recursive with once and it was just a quick demo, but my understanding is update is extremely expensive.

Re: Graph Databases 101

#69
post #43

Earlier quoted context omitted.

> a database which can efficiently represent graphs and offers functionality for traversing them First part is matching the commonly accepted definition (the one that had been around for about 50 years). The second part is your own invention. > This is not what gun does I did not even have a chance to take a look at that product yet. So far I'm just puzzled by the graph database definition some people seem to be usin…

I only just discovered graph databases a few years ago, so I have no idea what the definition of "graph database" was in ye olde days, but the parent's definition is the only one I'm familiar with. And it makes the most sense. After all, as programmers we're rarely concerned about the layout of data in memory, but rather the abstract data type (ADT) that we have to work with. An ADT is defined not by it's memory layo…

A typical graph dbms exposes a node selection (often only allowing to start at a single entry node or selecting a set of nodes by tags) and arc selection / filtering. Given that they were mostly used for CADs such an interface makes a lot of sense.

None of such databases ever featured a query language capable of defining a Dijkstra algorithm.

And, no, for a typical use of a graph database, it matters most how cheap it is to follow a graph arc. Therefore, representation matters. Otherwise a graph interface on top of a relational storage would have been sufficient.

Re: Graph Databases 101

#70
post #46
post #43

Earlier quoted context omitted.

> a database which can efficiently represent graphs and offers functionality for traversing them First part is matching the commonly accepted definition (the one that had been around for about 50 years). The second part is your own invention. > This is not what gun does I did not even have a chance to take a look at that product yet. So far I'm just puzzled by the graph database definition some people seem to be usin…

perhaps this is a terminology issue, I'll admit that I'm too young to have used pre-relational graph databases, but presumably they have a way of navigating / jumping between documents/vertices, (presumably based on pointers), otherwise what point would there be in having a graph? I'd encourage you to look at the product and see whether it meets your definition.

Yes, of course you could always select a node (often you'd always have to start from a single root node), select arcs, filter the arcs by some criteria, etc.

But I've never seen a complex query language that would allow to express any complex traversal strategies (like Dijkstra algorithm), and from your wording I concluded that this was your requirement for something to be called a graph database.

Post reply on HN