Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cassandra Performance

datastax.com

Post reply on HN
Print view View original

61–70 of 76 posts

Re: Cassandra Performance

#61

Post by rescrv »

tylerhobbs wrote: post #47
rescrv wrote: post #34

Earlier quoted context omitted.

Check out HyperDex, Hbase and BigTable for systems that provide better guarantees than "eventually."

You would need to define "consistency" in order to have a more reasonable discussion about what each system provides, but Cassandra certainly isn't only eventually consistent. You can choose the number of replicas that must respond in order to consider the read/write operation a success per operation, which allows you to have quorum-based strong consistency guarantees. There are more details on the options here: http…

Consistency is a safety property. HyperDex, Hbase and BigTable all provide linearizability, which has a well-defined meaning. Cassandra does not, and most of its descriptions of consistency only refer to the behavior of the system, and not the properties you can rely upon. Pointing to the number of replicas read or written only clouds the issue.

Re: Cassandra Performance

#62

Post by rescrv »

parasubvert wrote: post #54
jmix wrote: post #35

Earlier quoted context omitted.

False -- if there are nodes being added or deleted from the system, Cassandra provides no guarantee of consistency. Two nodes might disagree on quorum membership and thus quorum accesses may fail to overlap, leading to inconsistency. The consistency claims are overblown.

Please enlighten us with a cluster database that enables guaranteed consistency with dynamic node membership. They all have quirks handling membership (unless you're looking at a shared disk setup).

Checkout HyperDex. We just released a new version, and it is indeed consistent as nodes join and fail.

Re: Cassandra Performance

#63

Post by rescrv »

corresation wrote: post #12
aoprisan wrote: post #3

Where's the MongoDB comparison? They mention it but don't see it in their graph

Aside from MongoDB not being a part of the subject study, it's also worth mentioning that they cherry picked the example that made Cassandra look particularly good. MySQL actually did extremely well on the non-scan tests, while offering consistency. It depends upon your usage.

When benchmarking HyperDex we found something similar. Cassandra is better at writes than reads by a surprising amount. Given Facebook's elaborate caching architecture (which likely pushed for such an inversion) this makes sense.

Re: Cassandra Performance

#64

Post by crypto5 »

rescrv wrote: post #61
tylerhobbs wrote: post #47

Earlier quoted context omitted.

You would need to define "consistency" in order to have a more reasonable discussion about what each system provides, but Cassandra certainly isn't only eventually consistent. You can choose the number of replicas that must respond in order to consider the read/write operation a success per operation, which allows you to have quorum-based strong consistency guarantees. There are more details on the options here: http…

Consistency is a safety property. HyperDex, Hbase and BigTable all provide linearizability, which has a well-defined meaning. Cassandra does not, and most of its descriptions of consistency only refer to the behavior of the system, and not the properties you can rely upon. Pointing to the number of replicas read or written only clouds the issue.

I think if you will write to Cassandra with consistency level ALL, you will get strong consistency. Or you can use write consistency = 1 and read = ALL, or write consistency = Quorum and read consistency = Quorum.

Re: Cassandra Performance

#65

Post by justin_hancock »

linuxhansl wrote: post #44
justin_hancock wrote: post #37

Earlier quoted context omitted.

My understanding of facebooks Pod Architecture for HBase was not the name node but simply scaling HBase, HBase gets rather unpleasant at facebook scales. The facebook HBase fork has things like compactions disabled to improve performance. I ran a HBase cluster with 1PB storage, it became very unwieldy at this scale, thousands of regions and lots of tricks to keep it happy. As for SPOF, the name node now has HA and it…

Interesting. Do you remember what kind of problems you ran into and what version of HBase you used?

HBase 90.4, problems with I/O we had a very heavy read load on top of a write load, write load bursting 14,000 TX per second, and an average of 8,000 per second - each record around 2k.

Because of the I/O the WAL had to be turned off, this introduced problems when Region Servers occasionally died. Implementation of large regions 10GB, and fairly large HBlocks 512MB, increasing flush sizes to reduce minor compactions. Use of MSLAB to virtually eliminate GC all together, use of large heap 12GB on RS.

Worst problems we experienced was META corruption, that really , really sucked.

Re: Cassandra Performance

#66

Post by linuxhansl »

justin_hancock wrote: post #65
linuxhansl wrote: post #44

Earlier quoted context omitted.

Interesting. Do you remember what kind of problems you ran into and what version of HBase you used?

HBase 90.4, problems with I/O we had a very heavy read load on top of a write load, write load bursting 14,000 TX per second, and an average of 8,000 per second - each record around 2k. Because of the I/O the WAL had to be turned off, this introduced problems when Region Servers occasionally died. Implementation of large regions 10GB, and fairly large HBlocks 512MB, increasing flush sizes to reduce minor compactions.…

Thanks. If there's a more detailed writeup you can point me to that'd be great. I would like to make sure then that all these issues are addressed in the current versions.

0.94+ has MSLAB by default, with HFileV2 (0.92+) we can support much larger regions (20G or bigger). Curious about the 512M HBlocks, did you have scan-heavy read-load?

14k TX peak per regionserver? x 2k that's 28M/s (56M with WAL). Should be doable now even with WAL (definitely with deferred flush). Well, maybe not with concurrent very heavy read load, depending on disk configuration.

Probably on top of Hadoop 0.20-append? Hadoop-2.x.x should be far better too.

Re: Cassandra Performance

#67

Post by linuxhansl »

parasubvert wrote: post #54
jmix wrote: post #35

Earlier quoted context omitted.

False -- if there are nodes being added or deleted from the system, Cassandra provides no guarantee of consistency. Two nodes might disagree on quorum membership and thus quorum accesses may fail to overlap, leading to inconsistency. The consistency claims are overblown.

Please enlighten us with a cluster database that enables guaranteed consistency with dynamic node membership. They all have quirks handling membership (unless you're looking at a shared disk setup).

It's all about the tradeoffs you are taking. Cassandra favors availability over consistency. For many workloads that is desired for others it is not.

HBase for example is always consistent, but if you lose a node unexpectedly some data is not available for some time.

Re: Cassandra Performance

#68

Post by linuxhansl »

henrikschroder wrote: post #42
meh01 wrote: post #31

Serious question: are people really still using Cassandra? I've only ever heard horror stories about big deployments, and the only posts about it come from DataStax.

Netflix is probably the most well-known large user currently.

As far as I know they still store all critical data in Oracle. (That is actually true for many user for NoSQL data bases, not just Cassandra or Netflix).

My information is a bit dated, so that might no longer be the case.

Re: Cassandra Performance

#69

Post by linuxhansl »

thezilch wrote: post #27
linuxhansl wrote: post #26

HBase looks bad in some of these benchmarks, because it is hard to setup and has many tuning knobs to be tuned correctly for the workload in question. Due to its strictly consistent nature you have to think about key design, hotspotting of servers, etc, etc. In return you get correct atomic operations, row transactions, range scans by default (Cassandra uses a random partitioner by default not allowing range scans),…

And more usually, you'll use the tool the tool that your most familiar with or can become most familiar [quickest]. You say so yourself, HBase is relatively hard to quickly come to terms. We can observe the same when viewing the "SQL camp" with MySQL and PostgreSQL (and MSSQL). It is also true of the Facebook HBase install! Completely unnecessary jab; the use of HBase was politically driven -- what the architects wan…

Hmm... Where to start.

First, kindly point at an Cassandra installation, which supports the size that HBase supports in this setting.

Can I assume you have first hand information about this decision from someone at Facebook? I was stating a fact... not a jab.

Re: Google. Nothing at Google is driven by an eventually consistent store. They got that part right from beginning. It's too hard to manage from the an application point of view (unless you store immutable data, in which eventual consistency is pretty awesome). Checkout BigTable, Megastore, Percolator, Spanner. Except for the latter these are all based on BigTable. Note that BigTable is always consistent like HBase (and unlike Cassandra).

Are you saying the MySQL is better than PostgreSQL (or vice versa?) because one is easier to setup than the other. That is a terrible way to pick a technology.

Out of the box functioning is nice, easy to get started, makes the product look good in benchmarks, etc, but it just hides the complexity until you deploy into a production setting.

Lastly, I agree that if you store the odd few TB here or there, you certainly do not need to bother with the complexity of HBase.

Re: Cassandra Performance

#70

Post by [deleted] »

parasubvert wrote: post #57
jmix wrote: post #39

Earlier quoted context omitted.

I have looked through the code to conclude that Cassandra's consistency claims are overblown.

I've looked at your user profile to suggest you're a troll account. Please elaborate on the observations you have that led to this conclusion.

[deleted]
Post reply on HN