Guide to MongoDB for startups

I worked at Wine Spectator for a year (2010-2011), http://www.winespectator.com/ . They had built their first web site circa 2000 using Oracle, Sun Solaris, and Vignette with Java templates. Circa 2009 they decided to scrap the old, expensive system and move to PHP/MySql and the Symfony framework. They could not decide what their new schema should be, so, in the name of keeping things flexible, they decided that all data would be in a single table. This table had 240 fields, most of them with generic names such as "modifier_01" and "modifier_02". This was an organization that was looking for the flexibility offered by MongoDB, but they tried to cram that flexibility into a relational database, and they did so by ignoring all the relational features offered by MySql. This "one size fits all" database table did not work for the last assignment I was given: import all the old FileMaker Pro databases to a system running Mysql/PHP/Symfony. I build an entirely different project, with its own database and schema. Lord knows who is maintaining it now.

Then I worked a year (2011-2012) at Shermans Travel, http://www.shermanstravel.com/ , which also tore apart its database. When I first arrived they were trying to save a system built with MySql and PHP and later forced to conform to the CakePHP framework. The database had over 300 tables, many of which were no longer in use. Of the tables that were in use, many had fields that were no longer in use. The code, and the database, were a sprawling mess, that had evolved chaotically. (I am emphasizing this chaos because this criticism is often made of MongoDB: without a schema then how do you keep your data organized? Well, most of the places I have worked have had relational databases where the data was completely disorganized). After a few months, the CTO and the tech team decided on a complete re-write of the code. The tech team was allowed to vote for either Java or Python or Ruby (no one wanted to use PHP). We voted for Ruby. We rebuilt the site as 6 apps, using MySql for some of the apps and MongoDB for some. My last big project there was a rescue effort for a broken group of 4 database tables in MySql. There was a "users_history" table that was suppose to track whether a user had subscribed or unsubscribed to various newsletters we offered, but there had been a bug, apparently for years, such that many of the "unsubscribe" attempts were not recorded. There were 3 other tables with somewhat redundant data, and I wrote a script that scanned those other 3 tables and attempted to funnel the correct data to the 4th table.

I have many more stories like this. I could write a whole book about places where Oracle, MySql or PostGre was in use, but the data was badly organized. Unused tables and unused fields are extremely common.

Why do I emphasize the chaos I have encountered? Because the charge of badly organized data gets thrown at MongoDB a lot. If you would like to read a scathing attack against MongoDB, read this:

http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...

But to me, this line of argument compares the platonic ideal of relational data against the actual use of MongoDB. Maybe if Edgar F. Codd designed your schema to the 7th Normal Form then your schema really is well organized, but I have not seen anything like this in real life.

What I have seen, in real life, convinces me that every organization has an informal schema that is constantly evolving, and which can not be maintained with anything like regularity. Most of the organizations I've been hired at default to rebuilding everything every 5 or 6 years, because by that point the old system has grown chaotic. Sarah Mei's description of the dangers of MongoDB matches my own experience with relational databases: "we figured out that we had accidentally chosen a cache for our database."

What I like about MongoDB is it openly, boldly declares that the chaos I've seen is typical, and it facilitates the evolution of the schema which is going to happen no matter what you do or say. Things evolve, often chaotically. A programmer has a great idea and works on it in 2007, another programmer takes over in 2008, the project starts as raw PHP, later is imported into Symfony, then is re-written in Ruby, then it is broken up into several small apps. Someone quits. The CTO is fired. Someone new starts working and, for the sake of simplicity, prefers doing as much as possible as a background task, using cron scripts. A year later someone joins and is disgusted with the profusion of cron scripts, they want everything organized around a message queue. A very good sysadmin joins the team at a time when most of the programmers are weak, the sysadmin re-writes many of the background scripts, but he prefers Perl for everything and he implements some data caching strategies that no one understands.

You may think that I am exaggerating the level of chaos I have seen. I have not worked for Facebook or Google or Apple and if you tell me that in those companies everything is well run and well organized, then I will believe you, as I have no reason to doubt you. But I have worked at a lot of older media companies in New York City, and what I have seen is constant churn, churn at every level, churn in the team, churn in the technologies, and churn in the database.

I know I will be misunderstood, so let me try to clarify this:

I am not saying that chaos is good.

I am not saying that MongoDB is good because it encourages chaos.

I am saying that chaos is a symptom of the fact that most businesses do not know what their schema should be, and even if they did know what their schema should be, their needs would be different a year from now. The real schema needs of the organization (that is, what sets of data should be acquired and what the relations should be between those sets) are undergoing constant evolution, and this evolution is necessary, healthy, and unstoppable.

What is the strength of a relational system? Consider Wikipedia's explanation of Codd's Theorem:

http://en.wikipedia.org/wiki/Edgar_F._Codd

"The domain independent relational calculus queries are precisely those relational calculus queries that are invariant under choosing domains of values beyond those appearing in the database itself. That is, queries that may return different results for different domains are excluded. An example of such a forbidden query is the query "select all tuples other than those occurring in relation R", where R is a relation in the database. Assuming different domains, i.e., sets of atomic data items from which tuples can be constructed, this query returns different results and thus is clearly not domain independent."

Clearly, this assumes that the relations among the data are known. The organizations that I work with have no real idea about what relations they want to establish among their data. They are in a permanent exploratory phase. I believe these organizations could be described as "pre Codd", but most of them have been "pre Codd" for decades, and they will always be "pre Codd". If you force them to specify relations among their data, you will get answers exactly as useful as these:

http://blog.jimmyr.com/Funny_student_Exam_Answers_13_2008.ph...

MongoDB is useful in this context. Start acquiring data. Don't pretend you know what your schema is. You do not know what your schema is. The schema is changing all the time anyway.

Is there a place for relational databases? Yes, because sometimes some parts of the business become steady for some length of time, and for that part of the business, capturing fixed sets of data, with fixed relations, is very useful. But we should not pretend that this situation holds where it does not. I am not convinced that this is even the general case, though there is an overwhelming tendency in computer science, and in business, to pretend that fixed-sets-with-fixed-relations is the general case. If you feel it is, then you have been working at places facing conditions far more steady than what I have seen, or perhaps you are simply considering a shorter time frame than I am.

(Also, some guy by the name of Zuckerberg made a TON of money with PHP.)

Also, the tool is still in its toddler years ... and it is constantly compared to tools that have been around years longer (PG/MySQL). Comparing it to other NoSQL databases is just as silly (Riak, Cassandra) because they are completely different, have different constraints and are solving different problems.

There aren't better choices ... there are just different tools that are better for different jobs. It would be like looking at your toolbox and chunking everything but your saw or your hammer. Too many people are looking for the "technology to rule them all", but that doesn't (and shouldn't) exist.

I suppose row-level locking is a necessity when you want to be able to scale vertically. By comparison, I understand that MongoDB doesn't even try to support vertical scaling so it makes sense to not bother with complex locking systems.

Personally, I'd prefer to have to option of scaling without being forced to increase the number of moving parts in my single most important system (the database)...but the MongoDB docs make the fair point that there's a ceiling to vertical scaling. It's probably higher than most people think though.

As an aside, I discovered recently that Informix supports byte-level locks for 'smart large objects' (namely CLOBs and BLOBs). Makes me wonder whether field level locking will appear some day.

Also, the various gotchas involved with maintaining MongoDB at scale meant I spent most of my team's time chasing down esoteric problems instead of serving customers.

But ultimately, trusting a database with decades of development behind it to handle something like a JOIN turns out to be a good idea.

In my experience.

Just because your small scale system is running okay doesn't mean MongoDB is great, just that it's okay for your application.