“The benchmark numbers are completely wrong for both databases”

Post by steego » Wed, May 27, 2015, 5:35 PM UTC

copsarebastards wrote: post #44

weddpros wrote: post #21

Well, the author cares enough about RethinkDB to test it, even if he's a mongodb fan, even if his first benchmark was wrong, he was right to publish it: you all helped him when you pinpointed the problems in his tests... Thanks you for that. I don't see any marketing here, just the "do your own benchmark" best practice, and the "share with community" best practice... Does it make it a perfect benchmark? No, but at le…

So basically you're saying that because he cares and because he tried, and because the comments were dumb, nobody should criticize him? Do you really not care about getting accurate results? Running benchmarks is an engineering practice. If you failed to get meaningful results, you failed. Yes, he cares, yes, he tried, yes, the comments are dumb, but he still failed. Sure, I'll give the guy kudos for trying, but I'm…

Post by timmaxw » Wed, May 27, 2015, 6:47 PM UTC

weddpros wrote: post #34

timmaxw wrote: post #30

Earlier quoted context omitted.

> the author has corrected the discrepancies since then As of the time I posted this comment, the blog post still seems to be comparing indexed MongoDB operations against non-indexed RethinkDB operations. Under those conditions I'd expect RethinkDB to be at least 1000x slower than MongoDB. The fact that he's finding that RethinkDB is only 3x slower than MongoDB makes me think that there are still other major problems…

Some mistakes in his benchmarks, among probably others: - I don't see any mongodb index creation, so mongodb is inserting with no index while rethinkdb is inserting with the index. That's probably why there's a gap between the two - there's no mongodb index, and rethinkdb queries do not make use of the index (this is probably why rethinkdb is not 1000x slower: both aren't using indexes) - the $in query should be last…

Post by copsarebastards » Wed, May 27, 2015, 8:19 PM UTC

JDDunn9 wrote: post #48

copsarebastards wrote: post #44

Earlier quoted context omitted.

So basically you're saying that because he cares and because he tried, and because the comments were dumb, nobody should criticize him? Do you really not care about getting accurate results? Running benchmarks is an engineering practice. If you failed to get meaningful results, you failed. Yes, he cares, yes, he tried, yes, the comments are dumb, but he still failed. Sure, I'll give the guy kudos for trying, but I'm…

You must be fun at parties...

Post by andrewflnr » Wed, May 27, 2015, 9:23 PM UTC

threeseed wrote: post #32

timmaxw wrote: post #30

Earlier quoted context omitted.

> the author has corrected the discrepancies since then As of the time I posted this comment, the blog post still seems to be comparing indexed MongoDB operations against non-indexed RethinkDB operations. Under those conditions I'd expect RethinkDB to be at least 1000x slower than MongoDB. The fact that he's finding that RethinkDB is only 3x slower than MongoDB makes me think that there are still other major problems…

Not sure why you are frustrated it's just a blog post by someone who was inexperienced with your product. At least he owned up to the mistakes and was willing to fix it. It's an opportunity for you to work with the guy to show him how to do it properly and write a blog post of your own. I would say that you probably should look at your API because I've never used a database that required me to explicitly define which…