Should'ev tested MongoDB too!
Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
11–20 of 38 posts
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#12Interesting, though I miss Riak, Postgres and also a benchmark of degradation effects after running the workload a few 10 or 100 times (due to on-disk / index fragmentation).
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#13In a system like Redis Cluster (or any other where clients talk with the right nodes directly) it is conceivable that you always see linear scalability as the number of nodes goes up. This fact is not captured by the setup of this benchmark.
If you think at it, in a real-world scenario if you have 100 database nodes, you likely also have not threaded clients, but N different processes running in M different computer systems, querying different nodes independently.
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#14Earlier quoted context omitted.
Even though Redis is moving from being a Memcached competitor to being more of a Cassandra competitor and MySQL is used by many of the world's largest websites (often as a glorified key-value store) they are odd choices. Would anyone really use Redis for 'big data' ?
None of them do auto-sharding and master-master replication. This is the domain dynamo/bigtable-like systems (elasticsearch, riak, cassandra, hbase, voldemort)
Actually AFAIK MySQL supports master-master and the Cluster version supports autosharding.
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#15Earlier quoted context omitted.
I think they wanted to discriminate between them on tasks where someone might reasonably decide for either of them.
I haven't seen those usecases yet. Either you want to scale over multiple systems or you don't. Either you have more data than RAM or you don't.
How about Hacker News, and more generally internet discussion forums?
Many discussion forums are SQL backed - but both Reddit and Digg are reported to use Cassandra, a NoSQL system.
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#16Earlier quoted context omitted.
I haven't seen those usecases yet. Either you want to scale over multiple systems or you don't. Either you have more data than RAM or you don't.
How about Hacker News, and more generally internet discussion forums? Many discussion forums are SQL backed - but both Reddit and Digg are reported to use Cassandra, a NoSQL system.
In this case Redis would be the odd man out.
Unless you want to do manual sharding and replication, MySQL also is targeted at a different use-case. I also don't think anybody would run a 40 node mysql master-master cluster
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#17It's refreshing to see a benchmark that is somewhat thorough and doesn't even make sweeping statements like "X is faster than Y". Everyone who's ever written a hello world benchmark? Learn from this.
Well they did say that Cassandra is better than all the others - especially when compared to HBase - on nearly all measurements (except for high write scenario latencies.) This makes the decision of Facebook to go with HBase for their new messaging platform back in 2010 all the more strange. Though that was two years ago so things might have changed in Cassandra's favor since then.
[deleted]
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#18It's refreshing to see a benchmark that is somewhat thorough and doesn't even make sweeping statements like "X is faster than Y". Everyone who's ever written a hello world benchmark? Learn from this.
Well they did say that Cassandra is better than all the others - especially when compared to HBase - on nearly all measurements (except for high write scenario latencies.) This makes the decision of Facebook to go with HBase for their new messaging platform back in 2010 all the more strange. Though that was two years ago so things might have changed in Cassandra's favor since then.
> This makes the decision of Facebook to go with HBase for their new messaging platform back in 2010 all the more strange.
Speed isn't everything to a database. AFAIK they chose HBase over Cassandra because of consistency guarantees: eventual consistency is a bad choice for a messaging platform.
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#19Earlier quoted context omitted.
Well they did say that Cassandra is better than all the others - especially when compared to HBase - on nearly all measurements (except for high write scenario latencies.) This makes the decision of Facebook to go with HBase for their new messaging platform back in 2010 all the more strange. Though that was two years ago so things might have changed in Cassandra's favor since then.
> This makes the decision of Facebook to go with HBase for their new messaging platform back in 2010 all the more strange. Speed isn't everything to a database. AFAIK they chose HBase over Cassandra because of consistency guarantees: eventual consistency is a bad choice for a messaging platform.
eventual consistency is a bad choice for a messaging platform
Strange you would mention that in the context of Cassandra, since it allows for per-read/write configuration of consistency, from "eventual" to "strong". You get exactly what you ask for with Cassandra, whether its availability or consistency.
AFAIK, HBase only supports strong consistency.
Re: Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis
#20VoltDB Engineer here. It seems like the authors made some unfortunate choices in the configuration and usage of VoltDB.
A limited number of synchronous loads are always going to present scalability problems and will do a poor job measureing the throughput achievable. Even if it takes 500usecs to round-trip a transaction from the client, that means each synchronous client can only do 2000 per second. To scale linearly to 12 nodes might require hundreds of these synchronous clients.
VoltDB is commonly used with many parallel synchronous clients, such as web front-ends, or asynchronous workloads, such as event feeds. Both of these workload scale very well, so long as there is enough parallelism.
They also used global, consistent, multi-partition transactions for the YCSB scan. I'm not sure if that is necessary, and it would also limit scalability.
I can't speak too directly to the work because the code and configuration doesn't seem to be public. Perhaps I'm mistaken.
It would have been nice if they had reached out to us at VoltDB to verify their configuration and client. They also could have contacted Andy Pavlo at the H-Store who has some experience with YCSB on H-Store.
Still, we can always learn from something like this. One thing we're working on in VoltDB is better performance for workloads that aren't designed to be parallel. This means more performance for synchronous clients and for clients that do global consistent reads. Another area we could improve is more prominent diagnostics to show you why your cluster isn't running faster. That information is all available in our system tables and management tools, but we could probably boil it down to a single status field telling you whether your cluster is starved on client requests, intercluster-transaction-agreement or slow procedures. In our experience, it's usually the first one. I'll go file a ticket now.