Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mongodb – not evil, just misunderstood

siddharth-ravichandran.com

Post reply on HN
Print view View original

11–20 of 53 posts

Re: Mongodb – not evil, just misunderstood

#11

Post by djur »

MongoDB isn't necessarily bad, but there's a number of valid reasons it's received a lot of bad publicity.

It was a high-profile representative of the NoSQL trend and that trend is experiencing a backlash. Since it has a relatively easy learning curve and a JavaScript-centric interface (JS also beng trendy), it got a lot more uptake by startups and inexperienced developers wanting to learn about NoSQL (or move away from the supposedly moribund relational model).

The ad-hoc querying makes it tempting to build object mappers around Mongo and write the kind of code you would with an ORM. A lot of NoSQL refugees did exactly that. You can get pretty far doing that before the caveats with that approach kick in. It's much harder to fall into that trap with something like Riak.

And then MongoDB itself has had some poorly chosen defaults and design decisions. These wouldn't be so high-profile if it wasn't for the other issues.

Re: Mongodb – not evil, just misunderstood

#12

Post by p4lindromica »

I think this article misunderstands the implications of the fire and forget write concern. Fire and forget means that mongo has acknowledged a write that may not be persisted to disk. It appears the author uses fire and forget for notes, which are acceptable to lose, and tracking information, which are not acceptable to lose.

The author states the only downside of fire and forget is that data may not be available on subsequent queries. While this may be true, this should not be what you are worrying about. The downside is that when your primary crashes, or becomes overloaded to the point where your slaves all are lagging significantly and you need to switch primaries, the data is inconsistent and you will lose that acknowledged write.

Re: Mongodb – not evil, just misunderstood

#13

Post by yummyfajitas »

The "no migrations" claim is a lie. Migrations still exist, they just happen at the time of reading:

    def parseMongoRow(jsonBlob):
        x = None
        if jsonBlob['date']
For expiring fields in a redis database, this isn't a big deal. Your code stinks between [time of migration, time of migration + ttl]. For a permanent datastore, ouch.

I also do not understand why they are using MongoDB for this. They describe a schema involving shipments, shippoints and orders, and one of these things does not occur without the other (an attempt at justifying the document based data model?). I.e., something like this:

    CREATE TABLE orders (...)

    CREATE TABLE shipments (
      order_id BIGINT REFERENCES orders(id) NOT NULL,
      ship_from_id BIGINT REFERENCES shippoints(id) NOT NULL,
      ship_to_id BIGINT REFERENCES shippoints(id) NOT NULL
    )
You can't have a shipment without a shippoint. Black magic!

They have several hundred shipments/day, and lets be generous and assume there are 100 updates/notes to a shipment. We are talking maybe 50,000 inserts/day (omfg big data, invest in a 1TB hard disk!) and it sounds like data that's considerably more important than an ad impressions or pageviews. (Well actually it fits in ram, so maybe the 1TB hard disk on a dedicated server is overkill.)

Also, consider the daily aggregate generator in 3 lines of SQL rather than 236 lines of javascript:

    SELECT date, carrier, zone, COUNT(id), SUM(price)
          FROM shipments
          GROUP BY date, carrier, zone;
I don't get it. How is mongodb even remotely the right tool for this job?

Re: Mongodb – not evil, just misunderstood

#14

Post by tilsammans »

I guess I am old-school but you are listing a bunch of reasons why MongoDB is not evil, yet each and every one of these reasons turns out to be extremely risky business. All of which simply do not apply with relational datastores. The thing I took away from your post is that had you used a 20-year-old relational datastore you would have 0 of your issues anyway.

> The advantages of schemaless documents are priceless. Not having to migrate is just one of the perks. Our schemas were largely in the form of Orders (having many) Shipments (going_from) ShipPoint (to) ShipPoint

You say priceless. I don't think it means what you think it means. A migration is costly but also pretty rare. I migrate PostgreSQL with 100k+ rows as a matter of routine, it's over before you know it. The schema you are using (orders have many shipments going from point to point) are easily expressed in a relational schema and once defined would hardly ever need to change, if at all during the lifetime of the application. So what if I need to add a column here or there. It won't matter at all. Do you have more than 100 million documents in MongoDB? I guess you don't. Even if you do, relational has that covered too.

> This doesn’t always have to be the case, though it significantly contributes to Mongodb’s fast writes.

What you are saying is that I need to change MongoDB in order to make it safe. Relational database are safe out of the box, no change necessary.

> We add a lot of Notes to each shipment [...] it doesn’t critically affect the business workflows of the application.

Say what?

You're fine with data, even notes, being lost? That is completely acceptable to you? I guess this is what shocks me most. You kids think it's normal to lose data, and consider storing a note to be optional or something. It baffles me. If a note is optional, why have it in the first place?

> They do but since most of the stuff is memory mapped

Translates to: you need to have your data in RAM. This does not scale at all. It doesn't even begin to scale to the level where MySQL was. TEN years ago.

> Here is a simplified snapshot

What follows is a class that is 236 lines long. Two hundred and thirty-six lines long. Dear sir, if this is your simplified code I fear what your actual production code looks like. If you committed that to one of my repos we would have a very serious talk. Also you would do this exactly once during your career at my company.

> I haven’t even touched upon the replication and sharding features that Mongodb offers which I will reserve for another post.

Which every relational store also offers.

> To summarise I feel Mongodb is awesome

Why is it awesome? You have only shown me why it is horrible. I have seen nothing that is awesome. Optional data persistence, needs huge amounts of RAM, complex application level code to deal with reports, this is all stuff that you can do better, faster and more reliable with a relational solution.

Re: Mongodb – not evil, just misunderstood

#15

Post by davidw »

codex wrote: post #2

For every positive article on MongoDB there are ten which are negative. Now that's a strong brand.

Yep - when articles about something are prefaced with things like "not really evil" or "not as bad as it's made out to be", it's a pretty sure sign that your brand is tarnished.

Re: Mongodb – not evil, just misunderstood

#16

Post by djrobstep »

yummyfajitas wrote: post #13

The "no migrations" claim is a lie. Migrations still exist, they just happen at the time of reading: def parseMongoRow(jsonBlob): x = None if jsonBlob['date'] For expiring fields in a redis database, this isn't a big deal. Your code stinks between [time of migration, time of migration + ttl]. For a permanent datastore, ouch. I also do not understand why they are using MongoDB for this. They describe a schema involvin…

Mongo also stores the same column/attribute names with every single row. 50000 inserts? 50000 (usually identical) sets of attribute names stored (compared to once for SQL). And to my knowledge they are stored uncompressed. For a Big Data database, it doesn't seem very good at efficiently storing big data.

Re: Mongodb – not evil, just misunderstood

#17

Post by smegel »

> Mongodb expects that your working set fits into RAM along with the indexes for your database.

I don't really get this. Surely any database system is going to be faster if all frequently updated/accessed pages fit into ram...what makes mongo special in this regard? Why does it degrade so badly when it has to access disk (beyond the obvious)?

Re: Mongodb – not evil, just misunderstood

#19

Post by arethuza »

yummyfajitas wrote: post #13

The "no migrations" claim is a lie. Migrations still exist, they just happen at the time of reading: def parseMongoRow(jsonBlob): x = None if jsonBlob['date'] For expiring fields in a redis database, this isn't a big deal. Your code stinks between [time of migration, time of migration + ttl]. For a permanent datastore, ouch. I also do not understand why they are using MongoDB for this. They describe a schema involvin…

I've seen similar problems with systems that have supported multiple versions of the same XML schema at the same time - rather than taking the hit of migrating existing data to new schemas you end up having every more complex application code that has to check where a particular value may be stored (because nobody, in my experience, every bothers to store what version of a schema a particular document used).

This is particularly bad if these changes are small and incremental as each one in isolation looks reasonable but the cumulative effect over a few years can be a complete nightmare.

Post reply on HN