How to choose an in-memory NoSQL solution: Performance measuring • Hacker News Board

Re: How to choose an in-memory NoSQL solution: Performance measuring

#21

Post by takeda » Thu, Dec 31, 2015, 6:32 AM UTC

frik wrote: post #19

Why is memcached under-performing? Is it maybe their driver? Or their benchmark? Shouldn't it be similar to Redis, if not a bit faster? It's a bit fishy that the leading one is by the guys who wrote the article, see https://news.ycombinator.com/item?id=10814318

Those tests don't look reliable.

For example I'm quite surprised that no one pointed it out, but if you look, the memcache its performance grows nearly linearly to number of threads. They stopped at 256 when it was about to take over redis.

Now if you look at workload A and B, workload A supposed to have 50/50 read/write while workload B was 95/5 read/write. You see that memcached performed terribly there. You would think that perhaps the access pattern is different but then if you look at rest of databases they perform closely to workload A. And memcached which was doing 15000req/s is doing 10000req/s on a workload which supposed to be less work.

This looks to me that performance issue is most likely not in memcached itself but in their test application, but we can't prove that because they did not release code they used for testing.

Re: How to choose an in-memory NoSQL solution: Performance measuring

#22

Post by rtsisyk » Thu, Dec 31, 2015, 7:37 AM UTC

_mikz wrote: post #13

> Tarantool is an open-source NoSQL database management system and Lua application server developed in my.com. The first version of Tarantool was released in 2008 and the newest version is 1.6.7. and later > Through all tests we executed, Tarantool showed the best result for the count requests per second and for many of tests latency values on any type of examined workloads. Therefore, we can decide that for most of…

tarantool.org is an open-source project which is supported by my.com and many other commercial customers. my.com is an independent U.S. company that choose products on competitive basis.

Anyway, the benchmark is fully open. If in doubt, you can always download disk images and re-run benchmark.

// Disclaimer: http://tarantool.org/ developer

Re: How to choose an in-memory NoSQL solution: Performance measuring

#23

Post by daschl1 » Thu, Dec 31, 2015, 8:22 AM UTC

Sometimes it is depressing how little effort people put in to run a level playing field when doing numbers like this. The couchbase SDK used is from Mid 2013, with little effort the numbers could've been better..

Re: How to choose an in-memory NoSQL solution: Performance measuring

#24

Post by anarazel » Thu, Dec 31, 2015, 10:18 AM UTC

Cieplak wrote: post #16

hoodoof wrote: post #11

Postgres is also a good in-memory NoSQL solution. Actually you can use anything really as an in-memory NoSQL solution such as ArangoDB, MySQL, MongoDB, RethinkDB or whatever.

Do you mean using postgres tablespaces backed by memory or do you mean using a FDW? If you do mean using a ramdisk tablespace, the postgres docs recommend not doing that. However, if you really want to do that, make sure you attach xlog (the WAL table) to that ramdisk tablespace as well, otherwise every transaction will still hit the disk.

Not the poster upthread. But if you have enough memory (shared_buffers) and use unlogged tables, the effect is going to be that both reads and writes pretty much only hit memory, no disk. Without going through the trouble of moving tablespaces or anything.

Re: How to choose an in-memory NoSQL solution: Performance measuring

#25

Post by cihangirb » Thu, Dec 31, 2015, 5:43 PM UTC

Hi folks, was this a test over a single machine? Also anyone know why the test used an unofficial version of couchbase (4047)?

Re: How to choose an in-memory NoSQL solution: Performance measuring

#26

Post by rvncerr » Mon, Jan 04, 2016, 3:05 AM UTC

aristus wrote: post #6

The hardware setup is not clear. Are you running YCSB client and the database on the same single A3 instance?

No, I ran two instances close to each other (in one DC).

For example, Tarantool and CouchBase were installed on nosql-1 instance and YCSB client on nosql-2 (There are links to *.vhd files in the article).

Re: How to choose an in-memory NoSQL solution: Performance measuring

#27

Post by rvncerr » Mon, Jan 04, 2016, 3:14 AM UTC

cihangirb wrote: post #25

Hi folks, was this a test over a single machine? Also anyone know why the test used an unofficial version of couchbase (4047)?

Yes, it is. When I was running the test, CouchBase 4.x hasn't been released.

Re: How to choose an in-memory NoSQL solution: Performance measuring

#28

Post by rvncerr » Mon, Jan 11, 2016, 9:32 AM UTC

aps-sids wrote: post #2

I'm quite surprised to see that there's no Aerospike in this comparison.

This is only a first try. I'm going to make another review of a large number of NoSQL solutions soon. Aerospike will be observed there.

Re: How to choose an in-memory NoSQL solution: Performance measuring

#29

Post by rvncerr » Tue, Jan 12, 2016, 12:06 PM UTC

frik wrote: post #19

Why is memcached under-performing? Is it maybe their driver? Or their benchmark? Shouldn't it be similar to Redis, if not a bit faster? It's a bit fishy that the leading one is by the guys who wrote the article, see https://news.ycombinator.com/item?id=10814318

Underperforming might be caused by my YCSB memcached driver... I'll revise it. Memcached utilized all CPUs during the benchmark.

P.S. All VM images are open, you can repeat the benchmark if you wish.

Re: How to choose an in-memory NoSQL solution: Performance measuring

#30

Post by rvncerr » Tue, Jan 12, 2016, 12:07 PM UTC

espeed wrote: post #14

Apache Geode is a distributed, in-memory database ( http://geode.incubator.apache.org/ ). See https://news.ycombinator.com/item?id=10596859

I will try it. Thank you!