Non-relational database for a million rows of data? Madness.
Why you should never use MongoDB (2013)
181–188 of 188 posts
Re: Why you should never use MongoDB (2013)
#182Earlier quoted context omitted.
I'm painfully aware of the "Lack of isolation and multi document atomic updates is one of the biggest difficulties" problem, as I encountered it when I tried working with MongoDB. (I chose MongoDB because I was working with very fluid requirements and needed a very flexible schema.) But that's not really what the article complains about! The context is that I met some of the Diaspora leads in the summer of 2010. At t…
> by restricting "joins" to only when it's okay to see old data. So what do you do when it's not OK to see old data? That's when it breaks down. There are definitely ways around the issues, but you have to make compromises in how you build the application. If you don't make those design decisions early, you find yourself doing it wrong. In describing large scale systems, a friend of mine said "Imagine a scenario that…
So, would you say that it's a solvable problem, but it's so time consuming or difficult that a traditional relational database is a more appropriate database for average developers? (Remember, the Diaspora guys were right out of college and inexperienced.)
It's as if MongoDB is great for prototyping, and great for lossy data at scale, but it can't hit the middle.
Re: Why you should never use MongoDB (2013)
#183Earlier quoted context omitted.
I'm talking about "durability" here in the context of the "D" in "ACID". Previously, Mongo had very serious problems in that are because its client would assume that any message sent to the outgoing socket buffer was persisted "well enough", which was an obvious untruth [1]. This was also one of the somewhat underhanded techniques they used to achieve their early benchmarks. As of version 3, they have defaulted their…
MongoDB was initially designed to beat other nosql systems in benchmarks, is what I take away from reading about it. Someone took issue with that and wrote an article. "MongoDB lies" and "is slow" were some of the claims made from your link #1. Now that these issues are gone, by having reasonable default settings for write concern and journaling, how well does MongoDB do in the benchmarks today? Rewgarding default se…
> Now that these issues are gone, by having reasonable default settings for write concern and journaling, how well does MongoDB do in the benchmarks today?
Reports differ by benchmark, but the answer can be summarized as "not well". From [1] above:
> MongoDB is now a lot slower compared to v2.0. On the industry-standard YCSB benchmark, MongoDB used to be competitive with Cassandra, as seen in the performance measurements we did when benchmarking HyperDex. Ever since the change, MongoDB can no longer finish the entire benchmark suite in the time allotted.
I'm not sure I'd call what they were doing "cheating" per se because I honestly don't think they understood what they were doing, but it's fair to say that even if performance has improved since those benchmarks were run, Mongo definitely doesn't have any secret sauce.
Re: Why you should never use MongoDB (2013)
#184Earlier quoted context omitted.
> by restricting "joins" to only when it's okay to see old data. So what do you do when it's not OK to see old data? That's when it breaks down. There are definitely ways around the issues, but you have to make compromises in how you build the application. If you don't make those design decisions early, you find yourself doing it wrong. In describing large scale systems, a friend of mine said "Imagine a scenario that…
So, would you say that it's a solvable problem, but it's so time consuming or difficult that a traditional relational database is a more appropriate database for average developers? (Remember, the Diaspora guys were right out of college and inexperienced.) It's as if MongoDB is great for prototyping, and great for lossy data at scale, but it can't hit the middle.
Mongo's problem isn't really the lack of schema or joins or anything. It's the mountain of false assumptions people make around its behavior. If you understand the limitations of the libraries and systems you use, you can solve most problems. The issue is that most people don't understand those limitations. Mongo gets an especially bad rap because it actively pushed misleading information as part of its sales pitch.
Re: Why you should never use MongoDB (2013)
#185Earlier quoted context omitted.
It seems to me that you're advocating changing implementation details (binary query protocol instead of SQL, map-reduce queries instead of sprocs) and claiming that it is somehow fundamentally different. But I don't see how that would be the case. How is a binary query protocol fundamentally different than calling a sproc? How is a map-reduce job in Javascript any different on the attributes you mentioned than ad-hoc…
Binary query protocol is just a tweak, but it's a tweak that most RDBMSes are missing, and it matters for some workloads. Switching to a model where you supply your own map-reduce or pipeline is a real shift, I think, from the DB as a framework that manages querying for you to more of a library/toolkit you can use to write your own computations. The indexed tables model is an incredibly effective compromise, but it's…
Huh, interesting. I wonder if this is down to us having used different RDBMSes?
Most of my experience has been with SQL Server, which I think does use a (mostly) binary wire protocol, at least when calling sprocs. It also has a pretty good query optimizer - I've spent a lot of time trying to beat it with hand-tuned queries and only come up with something better about 50% of the time. And there are a lot of tools for source controlling the DB scripting and testing everything. It's not hard to do unit test style runs that set up and tear down tables and DBs.
Has that not been your experience?
Re: Why you should never use MongoDB (2013)
#186Earlier quoted context omitted.
Binary query protocol is just a tweak, but it's a tweak that most RDBMSes are missing, and it matters for some workloads. Switching to a model where you supply your own map-reduce or pipeline is a real shift, I think, from the DB as a framework that manages querying for you to more of a library/toolkit you can use to write your own computations. The indexed tables model is an incredibly effective compromise, but it's…
Huh, interesting. I wonder if this is down to us having used different RDBMSes? Most of my experience has been with SQL Server, which I think does use a (mostly) binary wire protocol, at least when calling sprocs. It also has a pretty good query optimizer - I've spent a lot of time trying to beat it with hand-tuned queries and only come up with something better about 50% of the time. And there are a lot of tools for…
> which I think does use a (mostly) binary wire protocol, at least when calling sprocs
Hmm. I'm used to invoking storedprocedures via "select mysproc(param1, param2, ...)" which still has to be formed into a string and then parsed on the DB side using the arbitrary-SQL parser (because there's no way for the DB to know a priori that it's not a "regular" query). Does SQL server have some special case binary protocol for invoking them directly?
> It also has a pretty good query optimizer - I've spent a lot of time trying to beat it with hand-tuned queries and only come up with something better about 50% of the time.
The query planner can usually run the best query possible with the indices that exist, sure. But you have to fit your calculation into this model of indices that are built on insert and everything else happening at query time. Or you go down the route of lazy materialized views that make use of other lazy materialized views, specifying indexing strategies... I mean I think you can ultimately express any data processing pipeline in a RDBMS if you try hard enough (though you have to use database-specific features that tend to be less-well supported by the ecosystem), but at some point it's easier to just have a first-class programming language that has access to the data, and write the code that you want to run.
> And there are a lot of tools for source controlling the DB scripting and testing everything. It's not hard to do unit test style runs that set up and tear down tables and DBs.
Up to a point. It's easy to end up with "unit" tests that take a second for each test, which mean it's not really practical to get good coverage of logic there.
In terms of source controlling and so on I guess the big problem is that you now have a distributed system written in two quite different technologies. So you've got to figure out a release and deployment process that handles both, and a lot of shops don't seem to bother. If you're already running a multi-language microservice architecture then this is probably a lot less of an issue.
Re: Why you should never use MongoDB (2013)
#187Earlier quoted context omitted.
Huh, interesting. I wonder if this is down to us having used different RDBMSes? Most of my experience has been with SQL Server, which I think does use a (mostly) binary wire protocol, at least when calling sprocs. It also has a pretty good query optimizer - I've spent a lot of time trying to beat it with hand-tuned queries and only come up with something better about 50% of the time. And there are a lot of tools for…
> which I think does use a (mostly) binary wire protocol, at least when calling sprocs Hmm. I'm used to invoking storedprocedures via "select mysproc(param1, param2, ...)" which still has to be formed into a string and then parsed on the DB side using the arbitrary-SQL parser (because there's no way for the DB to know a priori that it's not a "regular" query). Does SQL server have some special case binary protocol fo…
TBH I've never really looked at how TDS (the SQLServer wire protocol) works. From a code level, in ADO.NET you would create a query object, set the type to "sproc", set the name of the sproc, and attach parameters as objects. I bet the sproc name is going across as text, and maybe the param names, but that's probably it.
>>So you've got to figure out a release and deployment process that handles both, and a lot of shops don't seem to bother.
Yeah, that's true. Setting up something along the lines of Rails' migrations isn't really that hard, but many don't bother.
Re: Why you should never use MongoDB (2013)
#188Earlier quoted context omitted.
What's an example of an application that is a good fit for MongoDB?
I actually don't have any experience, but I heard this from wise people: If you have a file store service, where you're storing relatively big files (big images, videos, etc) and some metadata associated to them which need to be queried. It's easier to implement than "traditional" solutions (metadata goes in a RDBMS and actual files go to some directory, NAS, or whatever). Also, apart from being a nicer solution from…
> Also, apart from being a nicer solution from a programmer's perspective, you'll be able to easily scale horizontally because it supports sharding.
Sharding file storage is pretty easy.
On the other hand, I'm not familiar with the MongoDB APIs, but I assume it doesn't support handing a socket file descriptor from the webserver over to MongoDB so Mongo can sendfile(2) the data directly from the kernel's page cache to the TCP socket for locally resident data.
With the files stored directly on disk and metadata in an RDBMS, your webserver can sendfile(2) those files that are permanently stored locally or cached to local disk, and act as a proxy for other shards. Extra context switches and coping your data one or two times more than necessary can add up quickly.