Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: If you've used a graph database, would you use it again?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 84 posts

Re: Ask HN: If you've used a graph database, would you use it again?

#51

Post by andrewstellman »

Jakob wrote: post #21
andrewstellman wrote: post #7

I've been using RDF and triplestores / RDF databases for the last half-decade, developing both front-end and back-end systems, and training many developers to work in RDF. If you're used to either relational databases or object-oriented design, it's a really different way of thinking about data. Just like OOP is really good for certain kinds of problems and models, and RDBMS is good for other kinds of problems and mo…

This is interesting, I always wanted to find a good use case for RDF but in the end RDBMS worked out fine. Sports events seem a good example. What made it easier for you in your example of basketball play-by-plays with RDF? Taking the first example https://github.com/andrewstellman/pbprdf#example-analyze-a-s... and translating it into an RDBMS approach seems rather straight forward: GameEvent (PersonA, EventType, Per…

> Is it because of easier schema changes later on like introducing "secondsLeftInPeriod"?

That's definitely an important benefit. RDF makes it really easy to introduce changes that not only don't break the existing schema, but can be entirely isolated or combined in queries.

Another thing that RDF makes easy is analysis that takes advantage of a graph -- using relationships with other players, shots, etc. What players have the highest percentage making 3-point shots in possessions immediately after a player on the other team missed a 3-point shot? Building queries you compare previous possessions, shots, quarters; players' relations to each other (e.g. performance players who were subbed in after previous teammates went scoreless for 3 possessions) -- these things are a lot easier to do in RDF than in SQL.

Obviously, there are many things that are easier to model in RDBMS and query in SQL than with RDF/SPARQL. Every tool has its uses.

Re: Ask HN: If you've used a graph database, would you use it again?

#52

Post by andrewstellman »

ar-jan wrote: post #29
andrewstellman wrote: post #7

I've been using RDF and triplestores / RDF databases for the last half-decade, developing both front-end and back-end systems, and training many developers to work in RDF. If you're used to either relational databases or object-oriented design, it's a really different way of thinking about data. Just like OOP is really good for certain kinds of problems and models, and RDBMS is good for other kinds of problems and mo…

Do you know if there's off-the-shelf software (GUI) to create/edit/explore your own RDF dataset? Or does it always involve building your own front-end?

I like WebVOWL for visualizing the RDF ontology. Here's the pbprdf ontology displayed in it: http://www.visualdataweb.de/webvowl/#iri=https://raw.githubu...

A few years ago I put together a quick GUI in C# to make it easier to run SPARQL queries: https://github.com/andrewstellman/sparql-explorer

I haven't found an RDF editor or visual tool that I like. Some people like Topbraid Composer: https://www.topquadrant.com/tools/modeling-topbraid-composer... (commercial, closed source)

Re: Ask HN: If you've used a graph database, would you use it again?

#53

Post by pbowyer »

mrjn wrote: post #48

There is a lot of stigma attached to graph DBs. Would it provide good performance? Should I ever use it as my primary database? Is my data ever safe with a graph DB? If we go beyond that, assuming there was one which provided great performance, data integrity and can be reliable as a primary database — then Graph DBs are just better. First, the schema and data modeling is incredibly simple. Our minds think in graph t…

> We were solving this problem with Google's knowledge graph where we had to fit movie dataset in DB. The film industry has so many roles (director, producer, actor, cinematographer, and so on), that having a table for each, with many times same person doing multiple roles, is just super fucking hard. With hundreds of such roles, each role being a table would be insane. Representing this information in graphs is a cakewalk in comparison. And this problem gets a lot worse if you then switch to the music industry, books and others (hence, the decision to be a knowledge "graph").

Have you had success with modelling temporal data in a graph? E.g. "Bob worked for WB from 1999-2006, Disney 2006-2008, then WB again. In 2013 she transitioned and is now called Anna". Thinking of a property graph engine, both relations and properties would need to be versioned.

It feels like a graph database should be a good fit, better than a RDBMS for sure, but I've pages of sketches on how to model history and come up with nothing workable. https://arxiv.org/abs/1604.08568 is the best paper I've found; but I haven't got anything working.

My interest is as an amateur archivist, as dumping a description like the above into a text field and displaying it back to people is less useful than being able to query it or show the changes over time. Especially when you want to link it with files or media for retrieval purposes.

Re: Ask HN: If you've used a graph database, would you use it again?

#54

Post by somehnreader »

moxious wrote: post #39
somehnreader wrote: post #32

Earlier quoted context omitted.

How is a graph database built under the hood? I know that a decent RDBMS (simplified) will consist of the following: - data in blocks organised with a block-size that the underlying filesystem likes - a cache for the most frequently used blocks - every index is a B-Tree with pointers to the blocks containing the tuples Then there are column stores as well as row stores, and for compression you might have some diction…

Graph databases are built a lot of different ways; for example Neo4j's architecture is very, very different than something like an RDF triple store, or datastax on top of cassandra. [Neo4j internals can be seen here]( https://www.slideshare.net/thobe/an-overview-of-neo4j-intern... )...it's a bit old but I think mostly still accurate. In graphs you have to persist nodes and edges, though you may partition nodes by lab…

Thanks, very helpful. I am just looking at it and will have a bit of a think about this later :)

Re: Ask HN: If you've used a graph database, would you use it again?

#55

Post by mauimauimaui »

andrewstellman wrote: post #50
conceptme wrote: post #34

Earlier quoted context omitted.

Is there a particular RDF store which you would recommend? It seems scaling is a bit of an issue with Apache Jena. It's very easy to bring to entire system to a crawl with certain sparql queries and enough data in the store.

A team I've been working with for years has had a lot of success with Blazegraph: https://www.blazegraph.com/ -- the marketing materials call it "ultra-scalable," and that actually turns out to be true. I haven't seen too many cases where specific queries will cause serious performance with the system. That said, we've done some work to prevent runaway queries (e.g. strict query timeouts, downstream systems that hand…

I believe BlazeGraph is the basis for Amazon Neptune, the graph DB AWS announced at ReInvent last year. https://aws.amazon.com/neptune/

Re: Ask HN: If you've used a graph database, would you use it again?

#56

Post by srd31 »

mrjn wrote: post #48

There is a lot of stigma attached to graph DBs. Would it provide good performance? Should I ever use it as my primary database? Is my data ever safe with a graph DB? If we go beyond that, assuming there was one which provided great performance, data integrity and can be reliable as a primary database — then Graph DBs are just better. First, the schema and data modeling is incredibly simple. Our minds think in graph t…

The film industry example makes sense. Partially duplicative tables make development confusing.

Do you still feel there is an advantage of graph over relational when we have a known schema and known relationships without deep recursive relationships. For example, an inventory tracking system, we have items, customers, deliveries,etc...? I like the idea of being able to throw some metadata onto any of those tables quickly during prototyping, but my gut feeling is that long term we run into the need to be more structured and explicit like we do with a relational DB. It reminds me somewhat of the tradeoffs with NoSQL DBs during development

Re: Ask HN: If you've used a graph database, would you use it again?

#57

Post by andrewstellman »

mauimauimaui wrote: post #55
andrewstellman wrote: post #50

Earlier quoted context omitted.

A team I've been working with for years has had a lot of success with Blazegraph: https://www.blazegraph.com/ -- the marketing materials call it "ultra-scalable," and that actually turns out to be true. I haven't seen too many cases where specific queries will cause serious performance with the system. That said, we've done some work to prevent runaway queries (e.g. strict query timeouts, downstream systems that hand…

I believe BlazeGraph is the basis for Amazon Neptune, the graph DB AWS announced at ReInvent last year. https://aws.amazon.com/neptune/

That's great! I've had a chance to talk to some of the engineering and management folks at BlazeGraph over the years, and they're a really solid group. It's really nice to hear about a deserving team get success.

Re: Ask HN: If you've used a graph database, would you use it again?

#58

Post by vinceguidry »

pikchurn wrote: post #46
gr__or wrote: post #31

I used Neo4j for a few side projects but my go-to is still PostgreSQL. The largest flaw I see with Neo4j (and probably other graph databases as well) is that it forces you to think of your entities as either vertices or edges and that line tends to be not as clear as you might expect. For example: (:Person)-[:BEFRIENDS]->(:Person) If we'd want to store a date with that relationship, Neo4j got your back, that's entire…

Maybe you want a hypergraph? http://www.hypergraphdb.org/

I wish this were a networked database that I could just slap on a Digital Ocean droplet and query from anywhere.

Re: Ask HN: If you've used a graph database, would you use it again?

#59

Post by voondo »

I used the graph part of ArangoDB for a recent project and I appreciated the flexible nature of the relationships between entities (being edges, so always n-n). For example, my customer did often change its mind about some critical parts of the business logic (and thus relations between entities) and it was a pleasure to update without rewriting too much code. Also the queries involving many relationships seems more powerful and simpler than in a RDBMS. Anyway, maybe not an universal solution but, as a web/mobile developper, I can't see the actual limitations with my daily use case.

Re: Ask HN: If you've used a graph database, would you use it again?

#60

Post by rambossa »

threepipeproblm wrote: post #47
rambossa wrote: post #36

Are GraphDBs ever the right move in terms of performance?

For things that can reasonably be done with RDBMS, probably not... see "Do we need specialized graph databases? Benchmarking real-time social networking applications" (PDF link https://event.cwi.nl/grades/2017/12-Apaci.pdf ) But it seems likely that for queries that differentiate graph databases, such as finding long, variable-length paths, that there are cases where they can excel.

thanks for that link!
Post reply on HN