Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

arangodb.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#41

Post by chucky_z »

BlackFingolfin wrote: post #20
creshal wrote: post #18

Earlier quoted context omitted.

> PostgreSQL performed very well Despite the fact that you crippled it by not using jsonb columns.

Why so hostile? Personally, I assume this is just an instance of them not being PostgeSQL experts. So, before jumping to conclusions (namely that they deliberatly skewed their test, despite being very open in the way they described the setup), I'll instead wait and see -- perhaps they'll explain why they used json instead of jsonb, or perhaps (better!) they can updat the test to also include jsonb.

I think his hostility comes from the fact that if you google just "postgresql json" the second hit is the official documentation that goes over jsonb right in the second paragraph.

http://www.postgresql.org/docs/9.4/static/datatype-json.html

You can also see that the very first link outside of the official documentation:

https://www.compose.io/articles/is-postgresql-your-next-json...

Explains the benefits of jsonb and how to use indexes.

If you're going to include a database, at the very least do 20 minutes of research on it. If that can't be given, then just don't include something.

That's like saying "I've just installed MongoDB and read the intro documentation page, I'm now going to benchmark it against a MySQL cluster, which I have years of experience with, and that I helped develop." (E.g.: they developed ArangoDB, so they should be experts in at least that, right?)

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#42

Post by JamesMcMinn »

baldfat wrote: post #19

I was kind of shocked how good PostgesSQL did. I still think PostgresSQL and MariaDB are a better tool for most jobs considered big data.

Postgres was actually somewhat crippled in these tests since they used json rather than jsonb for storage, which stores the json in a binary format which doesn't need to be serialised on reads.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#43

Post by merlincorey »

amirouche wrote: post #35
ilaksh wrote: post #32

Why not include redis or rethinkdb?

redis and rethinkdb are not ACID across documents. So it's not the same usecase at all .

Are you implying that MongoDB and friends are ACID across documents?

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#44

Post by kbenson »

I wonder why there's not the equivalent to the Frameworks Benchmark[1] for databases. It seems we could all really benefit from that. Ideally it would get to a place where they would be able to simulate real-world worst case scenarios and test for problems. Each database would likely want multiple entries with different configs, but if you have some engineered failure scenarios and tests in the results it becomes obvious what the trade-off is. Sure, a specific setting may reduce consistency in the event of a failure for speed, but sometimes that's what you might want, and if the failure cases clearly show the problem, at least you aren't going in blind.

1: https://www.techempower.com/benchmarks/

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#45

Post by don71 »

I'm Claudius, author of the tests. I've been asked to include a lot of different databases into the test runs. The most requested databases were Postgres/JSON and RethinkDB. I started with Postgres. The Postgres manual states that JSONB might be faster, but some StackOverflow answers indicate that it takes more space than JSON, while JSON might be slightly more compatible with legacy code. I've shown the queries and setup to some local Postgres users. They did not point that JSONB will be much faster for the kinds of requests used in the test setup. For instance, we do not use special indexes apart from the primary one by choice.

I wanted to move on to RethinkDB next, but I see your point that a comparison between the different JSON formats of Postgres can also be very enlightening. This should replace guessing with hard facts. As always I will update the blog post and add this tests as well - as we did in the past, see https://www.arangodb.com/nosql-performance-blog-series/.

If you have any improvements concerning the configuration of Postgres or SQL queries, I'm will be more than happy to include them as well in the update. I will push the used configuration to GITHUB as well.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#46

Post by SilasX »

andy_ppp wrote: post #9
creshal wrote: post #7

Earlier quoted context omitted.

The particular benchmark seems bullshit, too. For postgres they seem to intentionally use the less performant json column type instead of jsonb. Not that I can verify it, because the code in the linked public "No magic, no tricks – check the code and make your own tests!" repository doesn't match the published results and doesn't even work at all with postgres… EDIT: Okay, they pushed a new version containing the Pos…

Just to be a pedant, the JSONB format as I understand is marginally slower at inserts and orders of magnitude faster at everything else...

Stupid question: isn't that the typical result of indexing? Slower writes for (much) faster lookups?

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#47

Post by creshal »

BlackFingolfin wrote: post #20
creshal wrote: post #18

Earlier quoted context omitted.

> PostgreSQL performed very well Despite the fact that you crippled it by not using jsonb columns.

Why so hostile? Personally, I assume this is just an instance of them not being PostgeSQL experts. So, before jumping to conclusions (namely that they deliberatly skewed their test, despite being very open in the way they described the setup), I'll instead wait and see -- perhaps they'll explain why they used json instead of jsonb, or perhaps (better!) they can updat the test to also include jsonb.

> Why so hostile? Personally, I assume this is just an instance of them not being PostgeSQL experts.

Postgres' official documentation on json and jsonb is rather concise: http://www.postgresql.org/docs/9.4/static/datatype-json.html

The difference between the types is described in paragraph two. So, they either benchmarked Postgres while having no idea whatsoever what they were doing, or they were deliberately crippling the competition.

Neither option is confidence inspiring.

(And the first option seems sketchy, seeing how they then went and re-created the whole benchmark as classical RDBMS setup for the second postgres test.)

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#48

Post by chucky_z »

don71 wrote: post #45

I'm Claudius, author of the tests. I've been asked to include a lot of different databases into the test runs. The most requested databases were Postgres/JSON and RethinkDB. I started with Postgres. The Postgres manual states that JSONB might be faster, but some StackOverflow answers indicate that it takes more space than JSON, while JSON might be slightly more compatible with legacy code. I've shown the queries and…

Please refer to the #postgresql channel on irc.freenode.net for any postgres inquiries, you will receive an answer from experts and core developers on the correct processes within minutes for almost any question. It is a very active channel full of knowledgeable folks.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#49

Post by lobster_johnson »

JamesMcMinn wrote: post #42
baldfat wrote: post #19

I was kind of shocked how good PostgesSQL did. I still think PostgresSQL and MariaDB are a better tool for most jobs considered big data.

Postgres was actually somewhat crippled in these tests since they used json rather than jsonb for storage, which stores the json in a binary format which doesn't need to be serialised on reads.

That's not quite correct. The jsonb requires that reads deserialize jsonb into textual JSON, whereas the json type can be sent directly to the client with no processing.

jsonb is superior when:

1. You want to use any of the built-in JSON functions, e.g. for extracting fields from the document.

2. You want to index the JSON (either the entire thing via GIN, or individual fields via ordinary B-tree indexes).

3. You want to save space; jsonb strips whitespace.

jsonb incurs an overhead on both reads and writes since it must serialize to/from textual JSON.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#50

Post by jerven »

I am just going to say: have a try with the LDBC social benchmark http://ldbcouncil.org/ and http://ldbcouncil.org/benchmarks. Where you can even have audited results.

These are also graph database benchmarks that are synthetic, designed to look like real data and are quite hard to do well on.

As someone responsible for a public free to use deployment of a graph database with more than 2 billion nodes and 15 billion edges (sparql.uniprot.org) I must say this looks like a SPARQL benchmark from 10 years ago.

Post reply on HN