Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing Database Types

prisma.io

Post reply on HN
Print view View original

121–130 of 176 posts

Re: Comparing Database Types

#121

Post by mrjn »

mdaniel wrote: post #95
planck01 wrote: post #84

Earlier quoted context omitted.

I am surprised Dgraph isn't mentioned as an example. It is the most starred graph db on Github, and I think it is the best one in terms of performance and scalability.

Strange that they have to have such a non-standard license, when they go out of their way to mention Apache 2 several times: https://github.com/dgraph-io/dgraph/blob/master/LICENSE.md Contrast that with Orient, who also have an Enterprise version, and they just straight-up say "Apache 2, no drama" https://github.com/orientechnologies/orientdb/blob/develop/l... We had an absolutely miserable experience trying to get J…

> write a big string literal and send it to the server

To be honest, that describes almost all query languages, including SQL and Cypher. Dgraph's QL is based on GraphQL[1]. When starting Dgraph, we had a choice to go with Gremlin or Cypher. But, we didn't like either choices -- Gremlin tends to be slower to execute, because of how they treat the queries, i.e. graph walks, with iteration over results to generate the next set of queries; something we explicitly avoid [2]. And both of them returned lists of things.

GraphQL, on the other hand, allows a subgraph to be returned. One can go from a subgraph to a list of things (lossy), but can't go the other way around, because you lose relationships (knowledge about how they are connected).

That's what led us to the path of GraphQL. Over time we modified it to what we have now, GraphQL+-. Meanwhile, over these last 3 years, GraphQL has become so much more popular than either Gremlin or Cypher, that now Neo4j is a founding part of the GraphQL Foundation, just like Dgraph is.

Now, Gremlin is still relevant in the graph space. So, we're planning to build Gremlin support. But, before that, we're working on building a native support for official GraphQL spec. That should let one integrate into the fast growing GraphQL ecosystem.

[1]: https://graphql.org/ [2]: https://blog.dgraph.io/post/why-google-needed-graph-serving-...

Re: Comparing Database Types

#122

Post by barrkel »

minitoar wrote: post #96
imchairmanm wrote: post #93

Earlier quoted context omitted.

Hello, author here. That's a good question and something I had a hard time sorting out as I worked on this. I think those fall into a different category confusingly sometimes called column-oriented databases. They're primarily used for analytic-focused tasks and get their name from storing data by column instead of by row (all data in a single column is stored to disk together). I didn't include those as a separate c…

That makes sense, they really are just relational databases optimized for certain tasks, with corresponding limitations e.g. they don't support arbitrary joins.

There's nothing intrinsic about not supporting joins, in a columnar store; it's just that you lose a huge amount of the linear scanning performance if you have to do joins for each value. Most columnar stores I've used (primarily Impala, SparkSQL and Clickhouse) all support joins, but they materialize one side of the join as an in-memory hash table, which limits the allowable size of the join, and is a cost multiplier for a distributed query. I believe per the docs that MemSQL can mix and match row-based with columnar more easily, but joins are always going to be really slow compared to the speed you can get from scanning the minimum number of columns to answer your question.

Re: Comparing Database Types

#123

Post by skybrian »

tlarkworthy wrote: post #61
taffer wrote: post #47

> Relational databases get their name from the fact that relationships can be defined between tables. This is a widespread misconception. Relational databases get their name from relations in the mathematical sense[1], i.e. sets of tuples containing facts. The basic idea of the relational model is that logical predicates can be used to query data flexibly without having to change the underlying data structures. The b…

What is the relation? The table? (I.e. the tuples describing the rows). Or is it the joins? (In which case the article is correct). The columns?

The value in a table at a particular time is a relation, but so is the value of a view, the result of a subquery, the result of a select statement, and the value represented by the contents of a CSV file. In some API's this is called a row set.

It's essentially a table-shaped value. Conceptually it's immutable, and relational algebra is about calculating new values from old ones. A select statement does this too.

Re: Comparing Database Types

#124

Post by namelosw »

danenania wrote: post #22
namelosw wrote: post #15

Earlier quoted context omitted.

Some databases design like extremely simple key-value databases cannot efficiently express joining relation unless load all the data in memory. The same could be said for column based databases etc. I guess that's quite a difference.

"key-value databases cannot efficiently express joining relation unless load all the data in memory" With the right design can't you store the graph data across multiple keys and then load pieces of it selectively into memory? I get that a graph db specializes in this pattern and makes it more efficient, easier to query, etc., but that doesn't mean it's the only type of db that can model a graph.

Technically it could be, just loading all the keys in a value, which could be massive.

The design of relational databases indicates the database is responsible to do a lot of logic according to the query language. Graph databases usually also include their own query language in order to do the same thing.

For example, from trillions of records there are 100 records with field x equals to value y. And the job is to get all the 100 records instead of trillions of them. The database would take a short query and interpret the predicate logic in the database process, instead of sending the data back to client which usually located in another physical machine.

Re: Comparing Database Types

#125

Post by cagmz »

ucarion wrote: post #36
asah wrote: post #28

Earlier quoted context omitted.

I've done this (commercially!) with PostgreSQL - just start with a single table, with one JSON field, and as you want performance, integrity, etc, add expression indexes, break out frequently used expressions into columns etc. On large tables, obviously there's a cost for this reorganization but you can partition the data first, and only reorg the most recent data (e.g. range partitioning by time). https://www.google…

Do you have any resources online for how well this approach works in practice? I've been thinking about doing this in replacing a MongoDB database.

Anecdotally, it's worked out well. Before storing a JSON object, we validate it at the app-level using a JSON schema (which has types, required keys, etc). This lets us write without worrying to much.

Once we felt that the schema wasn't changing as much, had too many concerns, etc, we made tables and wrote to them (instead of the JSON column).

Re: Comparing Database Types

#126

Post by kthejoker2 »

namanyayg wrote: post #81
Pheebau wrote: post #69

Earlier quoted context omitted.

A join in an intersection in relational algebra

Not necessarily, it can be a union too? Depends on the type of join.

A union is not a join at all - a union is a union. They're different concepts entirely.

Re: Comparing Database Types

#127

Post by kthejoker2 »

einpoklum wrote: post #112

The document completely overlooks Columnar databases, which are focused on analytics and are much faster than (most, not all) general-purpose DBMSes. See: https://en.wikipedia.org/wiki/Column-oriented_DBMS and https://www.slideshare.net/arangodb/introduction-to-column-o... or get: http://www.nowpublishers.com/article/Details/DBS-024 Examples: * MonetDB * SAP Hana * Actian Vector (formerly Vectorwise) * Oracle In-Memo…

Also Druid, HBase, Vertipaq (engine behind PowerBI), Redshift, Azure SQL DW, etc

Columnar compression is a really interesting engineering problem

Re: Comparing Database Types

#128

Post by kthejoker2 »

planck01 wrote: post #83
gibsonf1 wrote: post #27

The discussion of graph dbs completely misses the semantic rdf graph approach and how that differs greatly from the property graph (which is discussed). So important is not having to have a custom schema for each application that does not communicate with any other app as opposed to using standard ontologies with relationships and classes that are known and allow interoperability between systems (Linked Data Platform…

Do you know of any successfully semantic RDF graph databases, I guess with OWL support? Because I personally don't. If not, it probably is rightfully too much an academic niche to be discussed in the article.

Stardog, MarkLogic, Virtuoso, AllegroGraph, and RDF4J all have commercial applications, but yeah in general semantic RDF is dying on the vine.
Post reply on HN