Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

arangodb.com

Post reply on HN
Print view View original

11–20 of 56 posts

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#11

Post by qaqy »

cwyers wrote: post #3

I really, really distrust these kinds of evaluations when they come from someone whose product is included in the comparison. Even if everything is above-board, they're not going to publish if it shows their product just completely sucks at it. That kind of publication bias makes these kinds of results a lot less trustworthy than independent benchmarks even if you assume the best of intentions from the people putting…

WTF with VCs investing into all this crap? There are very few scenarios were you would be better off with NoSQL solution and there are established players serving those niches already.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#12

Post by robconery »

andy_ppp wrote: post #9
creshal wrote: post #7

Earlier quoted context omitted.

The particular benchmark seems bullshit, too. For postgres they seem to intentionally use the less performant json column type instead of jsonb. Not that I can verify it, because the code in the linked public "No magic, no tricks – check the code and make your own tests!" repository doesn't match the published results and doesn't even work at all with postgres… EDIT: Okay, they pushed a new version containing the Pos…

Just to be a pedant, the JSONB format as I understand is marginally slower at inserts and orders of magnitude faster at everything else...

It's fractionally slower, true, because of the serialization hit (string to binary). The real juice comes from the GIN index - and if you apply it to specific columns instead of a complete document, you have a rocket ship on read.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#13

Post by ifcologne »

cwyers wrote: post #3

I really, really distrust these kinds of evaluations when they come from someone whose product is included in the comparison. Even if everything is above-board, they're not going to publish if it shows their product just completely sucks at it. That kind of publication bias makes these kinds of results a lot less trustworthy than independent benchmarks even if you assume the best of intentions from the people putting…

Ingo from ArangoDB here. I agree that vendor tests are always biased, of course you want to show that your product is competitive.

But as there is no independent institution that compared our product and as we want to know where we stand with ArangoDB, Claudius did his own tests. And as the work is already done, why not share it.

We tried our best to do it as open as possible. PostgreSQL performed very well and we have a problem with memory consumption - have a look at the charts, we will try to improve there.

- Every database configuration is public

- All test scripts are available on Github

- We publish updates if we get pull-requests or comments with suggestions for improvements

We did that before and after the last test, some database vendors sent us improved snapshots of their databases which found their way into the latest products (OrientDB and Neo4j).

If you have suggestions for improvements, please let us know.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#16

Post by creshal »

robconery wrote: post #12
andy_ppp wrote: post #9

Earlier quoted context omitted.

Just to be a pedant, the JSONB format as I understand is marginally slower at inserts and orders of magnitude faster at everything else...

It's fractionally slower, true, because of the serialization hit (string to binary). The real juice comes from the GIN index - and if you apply it to specific columns instead of a complete document, you have a rocket ship on read.

> It's fractionally slower, true, because of the serialization hit (string to binary).

And with JSON columns you have to serialize on accesses, which is a lot slower in the read-mostly tests.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#17

Post by [deleted] »

ifcologne wrote: post #13
cwyers wrote: post #3

I really, really distrust these kinds of evaluations when they come from someone whose product is included in the comparison. Even if everything is above-board, they're not going to publish if it shows their product just completely sucks at it. That kind of publication bias makes these kinds of results a lot less trustworthy than independent benchmarks even if you assume the best of intentions from the people putting…

Ingo from ArangoDB here. I agree that vendor tests are always biased, of course you want to show that your product is competitive. But as there is no independent institution that compared our product and as we want to know where we stand with ArangoDB, Claudius did his own tests. And as the work is already done, why not share it. We tried our best to do it as open as possible. PostgreSQL performed very well and we ha…

[deleted]

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#18

Post by creshal »

ifcologne wrote: post #13
cwyers wrote: post #3

I really, really distrust these kinds of evaluations when they come from someone whose product is included in the comparison. Even if everything is above-board, they're not going to publish if it shows their product just completely sucks at it. That kind of publication bias makes these kinds of results a lot less trustworthy than independent benchmarks even if you assume the best of intentions from the people putting…

Ingo from ArangoDB here. I agree that vendor tests are always biased, of course you want to show that your product is competitive. But as there is no independent institution that compared our product and as we want to know where we stand with ArangoDB, Claudius did his own tests. And as the work is already done, why not share it. We tried our best to do it as open as possible. PostgreSQL performed very well and we ha…

> PostgreSQL performed very well

Despite the fact that you crippled it by not using jsonb columns.

Re: NoSQL Benchmark Compares PostgreSQL, MongoDB, Neo4j, OrientDB and ArangoDB

#20

Post by BlackFingolfin »

creshal wrote: post #18
ifcologne wrote: post #13

Earlier quoted context omitted.

Ingo from ArangoDB here. I agree that vendor tests are always biased, of course you want to show that your product is competitive. But as there is no independent institution that compared our product and as we want to know where we stand with ArangoDB, Claudius did his own tests. And as the work is already done, why not share it. We tried our best to do it as open as possible. PostgreSQL performed very well and we ha…

> PostgreSQL performed very well Despite the fact that you crippled it by not using jsonb columns.

Why so hostile? Personally, I assume this is just an instance of them not being PostgeSQL experts. So, before jumping to conclusions (namely that they deliberatly skewed their test, despite being very open in the way they described the setup), I'll instead wait and see -- perhaps they'll explain why they used json instead of jsonb, or perhaps (better!) they can updat the test to also include jsonb.
Post reply on HN