Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Year with MongoDB

blog.engineering.kiip.me

Post reply on HN
Print view View original

51–60 of 153 posts

Re: A Year with MongoDB

#51

Post by jasonmccay »

ericflo wrote: post #9
cheald wrote: post #8

Earlier quoted context omitted.

I think the article is a fair criticism, and I think your response is likewise fair. Mongo was built with horizontal scaling in mind, and to that end, it tends to suffer noticeably when you overload a single node. Things like the global write lock and single-threaded map/reduce are definitely problems, and shouldn't be pooh-pooh'd away as "oh, just scale horizontally". Uncompacted key names are a real problem, and a…

Mongo was not designed with horizontal scaling in mind. Riak, Cassandra, HBase, Project Voldemort...these are the projects that were designed with horizontal scaling in mind (as evidenced by their architectures.) But not Mongo.

I have to respectfully disagree. Your comment is a bit sweeping and I really don't think that MongoDB's sharding solution is a bad one ... simply the strategies are different.

There is a large set of nice features that makes Mongo, for most people, nice to use long before you even need to address sharding. The percentage of people that will need to shard is much lower than the percentage of people that can get considerable benefits of how you can query data in MongoDB, for example, vs. Riak.

You are such a Riak-lover. :P

Re: A Year with MongoDB

#53

Post by lalmalang »

We had a very similar situation ~300 writes per second on AWS. but I suspect some of this has to do with the fact that most people address scaling by adding a replica set, rather than the much hairier sharding setup (http://www.mongodb.org/display/DOCS/Sharding+Introduction), this seems natural b/c mongodb's 'scalability' is often touted. In reality though, because of the lock, RS dont really address the problem much, and we encountered many of the problems described by the OP.

Not to denigrate the work the 10gen guys are doing -- they are obviously working on a hard problem, and were very helpful, and the mms dashboard was nice to pinpoint issues.

We decided to switch too though in the end, though i still enjoy using mongo for small stuff here and there

Re: A Year with MongoDB

#54

Post by j-kidd »

spidaman wrote: post #28
geoffeg wrote: post #23

Earlier quoted context omitted.

DB-level locking is planned for MongoDB 2.2 which should be out within a few months. https://jira.mongodb.org/browse/SERVER-4328

Meh, if your other option is to use PostgreSQL and get row level locks, a db level lock is still a fail.

And here's a great post that provides some insight on how much effort has been put in by RDBMS vendors to handle locking:

http://stackoverflow.com/a/872808

Re: A Year with MongoDB

#55

Post by disbelief »

lalmalang wrote: post #53

We had a very similar situation ~300 writes per second on AWS. but I suspect some of this has to do with the fact that most people address scaling by adding a replica set, rather than the much hairier sharding setup ( http://www.mongodb.org/display/DOCS/Sharding+Introduction ), this seems natural b/c mongodb's 'scalability' is often touted. In reality though, because of the lock, RS dont really address the problem mu…

Mind sharing what you switched to? Another schemaless data store, or a more traditional RDBMS?

Re: A Year with MongoDB

#56

Post by corford »

justinsb wrote: post #26
antirez wrote: post #11

Earlier quoted context omitted.

You are wrong about Redis. Cheers

Care to explain? I believe for Redis, "appendfsync everysec" is the default. The poster's point was that MySQL and Postgres both ship with something like "appendfsync always", and you have to opt-in to the the less safe mode if you want to get more performance. Redis ships with the less safe mode pre-selected, and so has higher performance out-of-the-box.

You're right. The Postgres equivalent to "appendfsync always" is "synchronous_commit = on". Which AFAIK is the default.

However, one of the nice things about redis is that even if you run "appendfsync everysec" you never run the risk of corruption. You're only risk is losing a maximum of 2 seconds worth of data.

If you missed it, there's a wonderful blog post by antirez covering all of this (and a lot more) here: http://antirez.com/post/redis-persistence-demystified.html

Re: A Year with MongoDB

#57

Post by scubaguy »

judofyr wrote: post #40
mitchellh wrote: post #15

Earlier quoted context omitted.

> Ugh, this sounds like a maintenance nightmare. How do you deal with adding extra field to the document? Do you ever feel the need of running on-the-fly migration of old versions? (But when you do, shouldn't running a migration for all documents a better idea?) Yes, we did on-the-fly migration as we loaded old data in. Doing full data migration was not really an option because querying from MongoDB on un-indexed dat…

So it's actually 100% the same as you would do with on-the-fly migrations in SQL: (1) Add column and add code moves the old data when you access it. Deploy. (2) Let it run for a while. Run a background job that migrates the rest (this might be done days or months later). (3) Remove the column and the custom code. The more I hear about "schemaless" the more I realize that it doesn't make any difference at all.

For a large table, adding a new column can be non-trivial, each record is updated when the new column is added.

Re: A Year with MongoDB

#58

Post by madworld »

siavosh wrote: post #41
bretthoerner wrote: post #39

Earlier quoted context omitted.

> I don't get the safety issue, 20 months and we haven't lost any user data. shrug Nobody loses any user data until they do.

This should be a deal breaker for any serious app. Does the performance hit of safe mode negate all other advantages of MongoDB?

That's most people's findings. If your dataset can fit in ram [1] and you don't care about your data being safe then there might be an argument for MongoDB. Once you care about your data, things like Voldemort, Riak, and Cassandra will eat Mongo's lunch on speed.

[1] But as Artur Bergman so eloquently points out, if your data can fit in ram, just use a native data-structure (http://youtu.be/oebqlzblfyo?t=13m35s)

Re: A Year with MongoDB

#59

Post by guywithabike »

aschobel wrote: post #38

We love MongoDB at Catch, it's been our primary backing store for all user data for over 20 months now. > Catch.com > Data Size: 50GB > Total Documents 27,000,000 > Operations per second: 450 (Create, reads, updates, etc.) > Lock % average 0% > CPU load average 0% Global Lock isn't ideal, but Mongo is so fast it hasn't been an issue for us. You need to keep on slow queries and design your schema and indexes correctly…

450 ops/sec is nothing.

What's your breakdown between the operation types, and what kind of hardware are you on?

Re: A Year with MongoDB

#60

Post by lalmalang »

disbelief wrote: post #55
lalmalang wrote: post #53

We had a very similar situation ~300 writes per second on AWS. but I suspect some of this has to do with the fact that most people address scaling by adding a replica set, rather than the much hairier sharding setup ( http://www.mongodb.org/display/DOCS/Sharding+Introduction ), this seems natural b/c mongodb's 'scalability' is often touted. In reality though, because of the lock, RS dont really address the problem mu…

Mind sharing what you switched to? Another schemaless data store, or a more traditional RDBMS?

For the moment to Cassandra but very tempted to look into hbase in more detail soon...
Post reply on HN