Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating from MongoDB to Cassandra

fullcontact.com

Post reply on HN
Print view View original

31–40 of 65 posts

Re: Migrating from MongoDB to Cassandra

#31

Post by mason55 »

roycehaynes wrote: post #27
DigitalSea wrote: post #3

When I read posts like this all but confirming MongoDB isn't the great product 10Gen make it out to be, I wonder how the heck MongoDB are still even relevant and then I remind myself of the fact that 10Gen have one of the best marketing and sales teams in the game at the moment. While MongoDB has improved greatly over previous versions, I can't help but feel if 10Gen put as much effort into improving their product as…

I think part of the lesson learned from reading the post is MongoDB is best when you start sharding from the git-go, especially if you know you're going to have pounds of data.

Rebalancing shards in MongoDB sucks, especially if you have any kind of traffic. Which means that even if you start out with just a few shards you either need to keep up with growing your number of shards so that none of them are ever more than ~30% full or else you're in for a painful reshard experience.

At least this is what happened to me and my encounters with MongoDB (nee 10gen) were unsuccessful in speeding up our resharding.

Re: Migrating from MongoDB to Cassandra

#32

Post by jbellis »

CptCodeMonkey wrote: post #13
DigitalSea wrote: post #3

When I read posts like this all but confirming MongoDB isn't the great product 10Gen make it out to be, I wonder how the heck MongoDB are still even relevant and then I remind myself of the fact that 10Gen have one of the best marketing and sales teams in the game at the moment. While MongoDB has improved greatly over previous versions, I can't help but feel if 10Gen put as much effort into improving their product as…

Knowing some of the FC people first hand, MongoDB did actually serve them fairly well for a substantial amount of time. Until they started hitting max limits, it didn't really make sense to move to c . Going straight to C or something like it would have been almost cargo-cultish ( eg If we build industrial strength, we will get industrial levels of traffic ).

It sounds like you're assuming that productivity and power are opposites. Four years ago or even two, Cassandra was much harder to develop against than MongoDB, but that's not the case today.

Re: Migrating from MongoDB to Cassandra

#33

Post by jbellis »

monkey26 wrote: post #22

This article caught my interest as I've been reading into Cassandra. But some previous research had me thinking that Cassandra works best with under a TB/node. Is SQL still better when you have really large nodes (16-32TB) and only really want to scale out for more storage? I'm currently humming along happily with Postgres, but some of the distributed features, and availability of Cassandra look really nice.

Cassandra 2.0 can handle 5TB per node easily, 10TB with some care. Best to scale out, not up.

That said, if someone else has already made the hardware choice for you, you can always run multiple C* nodes on a single machine. I know several production clusters that fit this description.

Re: Migrating from MongoDB to Cassandra

#34

Post by rdtsc »

DigitalSea wrote: post #3

When I read posts like this all but confirming MongoDB isn't the great product 10Gen make it out to be, I wonder how the heck MongoDB are still even relevant and then I remind myself of the fact that 10Gen have one of the best marketing and sales teams in the game at the moment. While MongoDB has improved greatly over previous versions, I can't help but feel if 10Gen put as much effort into improving their product as…

> 10Gen have one of the best marketing and sales teams in the game at the moment.

No doubt. I have 2 MongoDB mugs even though I vowed to never touch a product produced by them again (it was, after the whole -- we'll throw your data write requests over the wall and pray fiasco).

Re: Migrating from MongoDB to Cassandra

#35

Post by krenoten »

monkey26 wrote: post #22

This article caught my interest as I've been reading into Cassandra. But some previous research had me thinking that Cassandra works best with under a TB/node. Is SQL still better when you have really large nodes (16-32TB) and only really want to scale out for more storage? I'm currently humming along happily with Postgres, but some of the distributed features, and availability of Cassandra look really nice.

It's much more about desired usage patterns than amount of storage. Cassandra and RDBMS's differ quite a lot in how you replicate, consistency guarantees, performant read patterns, performant write patterns, how you handle recovery, etc... If you intend to bring anything to scale it helps to understand the strengths and weaknesses of the underlying architecture.

Re: Migrating from MongoDB to Cassandra

#36

Post by krenoten »

jchrisa wrote: post #10

Viber, one of the largest over the top messaging apps, recently shared their conversion from Mongo to Couchbase. They ended up requiring less than half of the original servers, and better performance. If you want to see a video of their engineer telling the story, it's available here: http://www.couchbase.com/presentations/couchbase-tlv-2014-co...

Usually people who get burned by hypedb think twice before making the same mistake again.

Re: Migrating from MongoDB to Cassandra

#37

Post by bdcravens »

rdtsc wrote: post #34
DigitalSea wrote: post #3

When I read posts like this all but confirming MongoDB isn't the great product 10Gen make it out to be, I wonder how the heck MongoDB are still even relevant and then I remind myself of the fact that 10Gen have one of the best marketing and sales teams in the game at the moment. While MongoDB has improved greatly over previous versions, I can't help but feel if 10Gen put as much effort into improving their product as…

> 10Gen have one of the best marketing and sales teams in the game at the moment. No doubt. I have 2 MongoDB mugs even though I vowed to never touch a product produced by them again (it was, after the whole -- we'll throw your data write requests over the wall and pray fiasco).

I have one from an event they did here. What's telling is that I'm in Houston - far outside the SF/SV magic bubble.

Re: Migrating from MongoDB to Cassandra

#38

Post by victorhooi »

mason55 wrote: post #31
roycehaynes wrote: post #27

Earlier quoted context omitted.

I think part of the lesson learned from reading the post is MongoDB is best when you start sharding from the git-go, especially if you know you're going to have pounds of data.

Rebalancing shards in MongoDB sucks, especially if you have any kind of traffic. Which means that even if you start out with just a few shards you either need to keep up with growing your number of shards so that none of them are ever more than ~30% full or else you're in for a painful reshard experience. At least this is what happened to me and my encounters with MongoDB (nee 10gen) were unsuccessful in speeding up…

I think it's fairly obvious but trying to re-balance any system that already has load issues is probably not a recipe for success.

This cropped up in the "Don't use Mongo" FUD from a few years back.

Eliot from MongoDb responded:

https://news.ycombinator.com/item?id=3202959

In an ideal world, you'd monitor and plan your capacity proactively - I don't think there's really any magic button for - "My system is at capacity - horizontally scale it now, with no downtime!".

Re: Migrating from MongoDB to Cassandra

#39

Post by Xorlev »

victorhooi wrote: post #38
mason55 wrote: post #31

Earlier quoted context omitted.

Rebalancing shards in MongoDB sucks, especially if you have any kind of traffic. Which means that even if you start out with just a few shards you either need to keep up with growing your number of shards so that none of them are ever more than ~30% full or else you're in for a painful reshard experience. At least this is what happened to me and my encounters with MongoDB (nee 10gen) were unsuccessful in speeding up…

I think it's fairly obvious but trying to re-balance any system that already has load issues is probably not a recipe for success. This cropped up in the "Don't use Mongo" FUD from a few years back. Eliot from MongoDb responded: https://news.ycombinator.com/item?id=3202959 In an ideal world, you'd monitor and plan your capacity proactively - I don't think there's really any magic button for - "My system is at capacit…

Cassandra certainly makes it a lot easier. We've doubled our cluster with no downtime. Even adding a non-double amount of nodes can be performed without any downtime (though slightly more impact).

Re: Migrating from MongoDB to Cassandra

#40

Post by olavgg »

jbellis wrote: post #33
monkey26 wrote: post #22

This article caught my interest as I've been reading into Cassandra. But some previous research had me thinking that Cassandra works best with under a TB/node. Is SQL still better when you have really large nodes (16-32TB) and only really want to scale out for more storage? I'm currently humming along happily with Postgres, but some of the distributed features, and availability of Cassandra look really nice.

Cassandra 2.0 can handle 5TB per node easily, 10TB with some care. Best to scale out, not up. That said, if someone else has already made the hardware choice for you, you can always run multiple C* nodes on a single machine. I know several production clusters that fit this description.

PostgreSQL can handle petabytes easily. However if you need to query a petabyte of data, then you need to rethink your solution. PrestoDB + Hive + Hadoop may be what you need.
Post reply on HN