A Year with MongoDB
blog.engineering.kiip.me
A Year with MongoDB
1–10 of 153 posts
Re: A Year with MongoDB
#2Not related to the article but the site: has anyone else been getting "connection interrupted" errors with tumblr recently? If I load a tumblr blog for the first time in ~24 hours the first and second page loads will result in connection interrupted, the 3rd and beyond will all load fine.
Re: A Year with MongoDB
#3From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe.
It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scaling should be more available.
Just seems to me people don't like how mongodb is built, but if used as intended I think mongodb performs as advertised. In most cases I don't think it's the tool, rather the one using it.
Re: A Year with MongoDB
#4This article is an excellent articulation of the strengths and (fixable) issues with mongoDB.
I like MongoDB a lot, and the improvements suggested would really strengthen the product and could make me more comfortable to use it in more serious applications.
Re: A Year with MongoDB
#5From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…
While I agree that you should definitely use your tools in the best way you're capable of, I think for most people there's a baseline expectation that if you save data in a database, that data will be safe (at the very least, recoverable) unless something happens like the server catching on fire.
Nearly every other major database has this as the default -- MySQL, PostgreSQL, CouchDB, and Berkeley DB to name a few. (Redis doesn't, but it's also very upfront about it, and does provide this kind of durability as an option from early on.) So when MongoDB breaks this expectation, and when asked to support it as an option, just says, "That's what more hardware is for," it's a pretty big turnoff.
Re: A Year with MongoDB
#6From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…
You're right that MongoDB is designed from the start for horizontal scaling, and the author could have better articulated his reluctance to add machines.
However, his suggestions would improve mongodb performance in all configurations, not limited to single node.
The mongodb guys did a great job simplfying the internal architecture and focusing on a great API to get the product out quickly. I'm confident they can make these sorts of internal improvements to broaden the use of mongodb.
Re: A Year with MongoDB
#7From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…
[Note: I wrote the blog post]
I'm not at all against horizontally scaling. However, I don't believe that horizontally scaling should be necessary doing a mere 200 updates to per second to a data store that isn't even fsyncing writes to disk.
Think of it in terms of cost per ops. Let's just say 200 update ops per second is the point at which you need to shard (not scientific, but let's just use that as a benchmark since that is what we saw at Kiip). MongoDB likes memory, so let's use high-memory AWS instances as a cost benchmark. I think this is fair since MongoDB advertises itself as a DB built for the cloud. The cheapest high-memory instance is around $330/month.
That gives you a cost per op of 6.37e-5 cents per update operation.
Let's compare this to PostgreSQL, which we've had in production for a couple months at Kiip now. Our PostgreSQL master server has peaked at around 1000 updates per second without issue, and also with the bonus that it doesn't block reads for no reason. The cost per op for PostgreSQL is 1.27e-5 cents.
Therefore, if you're swimming in money, then MongoDB seems like a great way to scale. However, we try to be more efficient with our infrastructure expenditures.
EDIT: Updated numbers, math is hard.
Re: A Year with MongoDB
#8From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…
I think the article is a fair criticism, and I think your response is likewise fair.
Mongo was built with horizontal scaling in mind, and to that end, it tends to suffer noticeably when you overload a single node. Things like the global write lock and single-threaded map/reduce are definitely problems, and shouldn't be pooh-pooh'd away as "oh, just scale horizontally". Uncompacted key names are a real problem, and a consequence of it is that Mongo tends to take more disk/RAM for the same data (+indexes) than a comparable RDBMS does. Maintenance is clunky - you end up having to repair or compact each slave individually, then step down your master and repair or compact it, then step it back up (which does work decently, but is time consuming!)
None of these are showstoppers, but they are pain points, especially if you're riding the "Mongo is magical scaling sauce" hype train. It takes a lot of work to really get it humming, and once you do, it's pretty damn good at the things it's good at, but there are gotchas and they will bite you if you aren't looking out for them.
Re: A Year with MongoDB
#9From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…
I think the article is a fair criticism, and I think your response is likewise fair. Mongo was built with horizontal scaling in mind, and to that end, it tends to suffer noticeably when you overload a single node. Things like the global write lock and single-threaded map/reduce are definitely problems, and shouldn't be pooh-pooh'd away as "oh, just scale horizontally". Uncompacted key names are a real problem, and a…
Mongo was not designed with horizontal scaling in mind. Riak, Cassandra, HBase, Project Voldemort...these are the projects that were designed with horizontal scaling in mind (as evidenced by their architectures.) But not Mongo.
Re: A Year with MongoDB
#10> Safe off by default
I think that is fixed now.
But this the single most appalling design decision they could have made while also claiming their product was a "database". (And this has been discussed here before so just do a search if you will).
This wasn't a bug, it was a deliberate design decision. Ok, that that would have been alright if they put a bright red warning on their front page. "We disabled durability by default (even though we call this product a database). If you run this configuration as a single server your data could be silently corrupted. Proceed at your own risk". But, they didn't. I call that being "shady". Yap, you read the correctly, you'd issue a write and there would be no result coming back acknowledging that at least the data made into the OS buffer. I can only guess their motive was to look good in benchmarks that everyone like to run and post results on the web. But there were (still are) real cases of people's data being silently corrupted, and they noticed only much later, when backups, and backups of backups were already made.