Live data from Hacker News

YouTrackDB is a general-use object-oriented graph database

github.com

51–60 of 73 posts

Re: YouTrackDB is a general-use object-oriented graph database

#51

I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?

I have had very bad experiences with graph dbs at scale. To the point where I will never again work on a project based on a graph db. YMMV, but I'm done with them forever.

[edit] To elaborate: tuning read and write performance is difficult, essentially everything must be indexed. This results in an explosion in data size--so data that might be X TB in csv format becomes maybe 10X when you consider all the indexes and bloat from the graph db's storage format. Which brings us to the topic of compression: there isn't any. Moreover, graph dbs are basically all quite immature compared to other db software, they haven't seen the test of production to the same degree, nor have they gotten the same attention in terms of bugfixes, performance tuning, etc. So they often have very sharp edges that you encounter under load. So if you like spending tons of money and time on zero value work, choose a graph db!

[another edit] Another problem with graph dbs is they encourage laziness in data modelling. I would echo others' recommendations here: just start with SQL. If it gets to the point where you actually need a graph query language--and you've already appropriately considered your life choices and determined that there's some value to what you're doing--only then should you consider trying to implement something on a graph query engine. But I'm not aware of one that is actually good. That's the problem. If you're at the point where you absolutely need a graph database you are probably at the point where you need to build your own. And you'll probably fail like all the other ones.

Re: YouTrackDB is a general-use object-oriented graph database

#52

I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?

Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases.

If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres and Neo4J, they will look identical, just that the leaf-nodes in Postgres will be a table-scan, while in Neo4J they will be either a vertex-scan or a edge-scan (which can both be seen as special cases of a table-scan).

As someone that has worked a lot with graph databases in the past, I'd largely recommend not using them. The price you pay in terms of worse ecosystem and less battle-tested maintenance tooling is not worth it to just have better syntax.

Re: YouTrackDB is a general-use object-oriented graph database

#53
post #5

Earlier quoted context omitted.

Neo4j is a great DB but their license price is egregious for enterprise customers. A few years ago I was involved in negotiating a contract for a small/medium size kubernetes deployment (think around 25 cores) and the annual price was more than the salary of a senior SWE full-time equivalent. See this page for an idea of their prices in 2018: https://blog.igovsol.com/2018/01/10/Neo4j-Commercial-Prices....

> small/medium size kubernetes deployment (think around 25 cores That's ~1 machine. 1 SWE for a database isn't egregious, databases provide huge value, but for that little performance, that's crazy. I can only assume as core count has blown up over the last 10 years, the pricing has somewhat diminished, but still, I'd be expecting a heck of a lot more capacity for 1 SWE.

I can tell you from quotes that I've seen, that compared to the 2018 prices listed on the page, end-of-2022 pricing was slightly up per core.

We were already well on our way of dumping Neo4J due to performance/operational/architectural reasons at the point, but seeing the quote solidified it.

Re: YouTrackDB is a general-use object-oriented graph database

#54
post #52

I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?

Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases. If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres a…

Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.

Re: YouTrackDB is a general-use object-oriented graph database

#55

I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?

Graph query languages like Cypher are great, but I am wholly unconvinced by the concept of a dedicated, general-purpose "graph database".

IMO, you're better off just using Postgres/etc, modeling your graph data there, and pulling in subsets of your graph for in-memory analysis. This is for the 99% of enterprises that aren't doing online streaming graph analysis on TB-scale graphs, and the other 1% should probably figure out something tailored* to their specific business model.

* Graph algorithms are more accessible than ever with GenAI code, and efficiently modeling a graph in memory is trivial (it's just structs with pointers to other edges/nodes, plus its nice to have full control over the memory layout).

Re: YouTrackDB is a general-use object-oriented graph database

#56
post #54
post #52

Earlier quoted context omitted.

Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases. If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres a…

Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.

AGE actually looks very interesting. When I evaluated it a while back it seemed like the project wasn't very active and I couldn't find any information about folks using it at scale in production. If I was forced to use a graph database again, and I didn't immediately quit upon receiving that news, I might give AGE a look.

Re: YouTrackDB is a general-use object-oriented graph database

#57
post #54
post #52

Earlier quoted context omitted.

Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases. If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres a…

Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.

Postgres 19, which is about to be released is adding support for SQL/PGQ.

Re: YouTrackDB is a general-use object-oriented graph database

#58
post #55

I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?

Graph query languages like Cypher are great, but I am wholly unconvinced by the concept of a dedicated, general-purpose "graph database". IMO, you're better off just using Postgres/etc, modeling your graph data there, and pulling in subsets of your graph for in-memory analysis. This is for the 99% of enterprises that aren't doing online streaming graph analysis on TB-scale graphs, and the other 1% should probably fig…

We had general-purpose graph databases before graph databases became a thing. I'm talking about relational databases. Values are the vertices, tuples are the edges, FK constraints are inclusion dependencies. In fact, n-ary relations means a hypergraph database, not just a binary graph db.

Re: YouTrackDB is a general-use object-oriented graph database

#60
post #54
post #52

Earlier quoted context omitted.

Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases. If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres a…

Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.

Unfortunately AGE is very immature and optimized even simple traversal queries rather poorly.
Post reply on HN