Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every Cassandra feature has some surprising behavior

blog.parsely.com

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Almost every Cassandra feature has some surprising behavior

#11

Post by frik »

dvirsky wrote: post #8
saurik wrote: post #7

Earlier quoted context omitted.

The one that is really irksome is the COMPACT STORAGE one, as it is explicitly disrecommended and deprecated by the Cassandra documentation. I have not yet gotten around to using Cassandra in any production environment, nor have I done any large-scale tests, and it has been a while since I delved deeply into the data model, but I remember looking at this part of the storage system and thinking it was "wrong", but won…

I haven't dealt with the ops side of running our clusters, and the details are not fresh in my mind, so I don't remember much about the storage engines. But we encountered surprises on almost any front when taking C* to production.

Given your experience: Have you considered a NewSQL db like MariaDB with one of its modern storage engines? Also Facebook moved from their Cassandra to Hadoop/Hive a long time ago (and the MySQL side was never touched).

Re: Almost every Cassandra feature has some surprising behavior

#12

Post by threeseed »

dvirsky wrote: post #5

The thing is that contrary to many other tools, you can't run Cassandra with almost any of its default settings (maybe besides ports). In most tools you'll need to tweak a few defaults, with Cassandra you need to thoroughly read the docs on every little configuration in the server and schema definitions (especially if you're working in multiple DCs), you'll always find a little surprise if you skim through it. It's a…

I have to disagree. But again given how scalable Cassandra is we could live in different worlds.

I've run multi-gigabyte data sets in Cassandra without any config changes. And when I responsible for a 40 node Cassandra cluster we didn't do any tweaks other than a few JVM settings here and there.

Cassandra I have to admit though is very sensitive to how you model and store your database. The whole tombstone saga is never a fun one to go through.

Re: Almost every Cassandra feature has some surprising behavior

#13

Post by threeseed »

smoyer wrote: post #6

I love that the article pokes fun at click-bait articles ... I'm not a Cassandra user but having used several other NoSQL solutions, I'm not at all surprised that this team, who dove into a new technology without caution, was surprised.

It's not just dove in without caution but without fully checking design decisions.

For example using Maps other than for a storing a handful of values has always been discouraged.

Re: Almost every Cassandra feature has some surprising behavior

#14

Post by dvirsky »

frik wrote: post #11
dvirsky wrote: post #8

Earlier quoted context omitted.

I haven't dealt with the ops side of running our clusters, and the details are not fresh in my mind, so I don't remember much about the storage engines. But we encountered surprises on almost any front when taking C* to production.

Given your experience: Have you considered a NewSQL db like MariaDB with one of its modern storage engines? Also Facebook moved from their Cassandra to Hadoop/Hive a long time ago (and the MySQL side was never touched).

We've moved most of our data to redis (which we were using way before cassandra), with a bit of classic MySQL for off-line data, and Redshift for analytics. Right now redis is not using cluster mode, but we'll switch to it eventually. It's a big move, but I find redis more predictable, and for our data sizes (10s of Gs), cheaper to maintain.

Re: Almost every Cassandra feature has some surprising behavior

#15

Post by dvirsky »

threeseed wrote: post #12
dvirsky wrote: post #5

The thing is that contrary to many other tools, you can't run Cassandra with almost any of its default settings (maybe besides ports). In most tools you'll need to tweak a few defaults, with Cassandra you need to thoroughly read the docs on every little configuration in the server and schema definitions (especially if you're working in multiple DCs), you'll always find a little surprise if you skim through it. It's a…

I have to disagree. But again given how scalable Cassandra is we could live in different worlds. I've run multi-gigabyte data sets in Cassandra without any config changes. And when I responsible for a 40 node Cassandra cluster we didn't do any tweaks other than a few JVM settings here and there. Cassandra I have to admit though is very sensitive to how you model and store your database. The whole tombstone saga is ne…

My biggest cluster was 12 nodes, actually a 3*4DCs cluster. I don't recall what the exact issues were, but it took months to stabilize this cluster, and it never performed as well as we'd wanted.

Re: Almost every Cassandra feature has some surprising behavior

#16

Post by easytiger »

threeseed wrote: post #12
dvirsky wrote: post #5

The thing is that contrary to many other tools, you can't run Cassandra with almost any of its default settings (maybe besides ports). In most tools you'll need to tweak a few defaults, with Cassandra you need to thoroughly read the docs on every little configuration in the server and schema definitions (especially if you're working in multiple DCs), you'll always find a little surprise if you skim through it. It's a…

I have to disagree. But again given how scalable Cassandra is we could live in different worlds. I've run multi-gigabyte data sets in Cassandra without any config changes. And when I responsible for a 40 node Cassandra cluster we didn't do any tweaks other than a few JVM settings here and there. Cassandra I have to admit though is very sensitive to how you model and store your database. The whole tombstone saga is ne…

> I've run multi-gigabyte data sets

WOW. You've run several GB of data through it? Wow.

Re: Almost every Cassandra feature has some surprising behavior

#17

Post by threeseed »

dvirsky wrote: post #9
zikzikzik wrote: post #4

This Team Used Apache Cassandra Without Reading The Fine Manual… You Won’t Believe What Happened Next “You honestly expected that adopting a data store at your scale would not require you to learn all of its internals?” These aren't even internals, these are basic facts about Cassandra.

I do believe Cassandra requires a much more intimate knowledge of its internals if you want to do anything serious with it - more so than any other database I've worked with.

I have to agree. But with the clarification that it isn't significantly more. Spend a few days and you will be an expert.

And I think it's worth noting that the "serious" things you would do on Cassandra more often than not you couldn't do on most other databases.

Re: Almost every Cassandra feature has some surprising behavior

#18

Post by threeseed »

easytiger wrote: post #16
threeseed wrote: post #12

Earlier quoted context omitted.

I have to disagree. But again given how scalable Cassandra is we could live in different worlds. I've run multi-gigabyte data sets in Cassandra without any config changes. And when I responsible for a 40 node Cassandra cluster we didn't do any tweaks other than a few JVM settings here and there. Cassandra I have to admit though is very sensitive to how you model and store your database. The whole tombstone saga is ne…

> I've run multi-gigabyte data sets WOW. You've run several GB of data through it? Wow.

That was the small one. The 40 node one had hundreds of terabytes.

Re: Almost every Cassandra feature has some surprising behavior

#19

Post by manishsharan »

Cassandra or most NoSQL databases seem to require a lot more internals knowledge than what a development or an ops team would have. Most sysops teams have a cadre of certified DBAs and administrators and they get on by without any surprises. The one thing Oracle got right was their certification program which ensures there are no gotchas -- most DBA can deliver smooth operations. I guess it is because these NOSQL products are relatively new, they require only ninja level experts to fiddle with the controls.

I would be interested in hearing more from someone who has use a third party managed cassandra service like Google' casandra product https://cloud.google.com/solutions/cassandra/. Did you still deep internal knowledge to use Cassandra ? This is important to know for me because my organization will not have the resources to manage Cassandra clusters but we need a Cassandra like store.

Re: Almost every Cassandra feature has some surprising behavior

#20

Post by manishsharan »

dvirsky wrote: post #14
frik wrote: post #11

Earlier quoted context omitted.

Given your experience: Have you considered a NewSQL db like MariaDB with one of its modern storage engines? Also Facebook moved from their Cassandra to Hadoop/Hive a long time ago (and the MySQL side was never touched).

We've moved most of our data to redis (which we were using way before cassandra), with a bit of classic MySQL for off-line data, and Redshift for analytics. Right now redis is not using cluster mode, but we'll switch to it eventually. It's a big move, but I find redis more predictable, and for our data sizes (10s of Gs), cheaper to maintain.

If you moved your data off Caasandra to Redis, I am going to assume you must be using a lot of Lua to maintain some relationships in data in Redis. How is the performance? I have been experimenting with Redis and my lua scripts are long and I have been wondering how they will fare under production level loads.
Post reply on HN