Why I Migrated Away From MongoDB • Hacker News Board

Re: Why I Migrated Away From MongoDB

#211

Post by einhverfr » Wed, Sep 19, 2012, 9:39 AM UTC

gregjor wrote: post #209

einhverfr wrote: post #190

Earlier quoted context omitted.

I suspect that a lot of NoSQL folks come to NoSQL from MySQL. As for this part: >"When you have multiple applications accessing the same database, each with their own implementation of consistency rules and their own internal idea of the data schema, come back and explain how that is more fun than using an RDBMS and being able to sleep nights." The obvious answer from the MySQL/NoSQL folks is "What? That's crazy! Mul…

I wasn't able to figure out how to post a comment/reply on Blogger, sorry. Thanks for pointing me to your article from the HN flame war about MongoDB. If only there were more useful articles like this and less flaming. You wrote "...in MySQL, the application is king and the db a humble servant." That is exactly my experience, and it leads not only to terrible databases but to piles of awful code written around terrib…

It's too late to post on your blog, unfortuanately (great article btw) but I think your article hits the nail on the head. A couple points I would make though is that I don't see anything wrong really with using an ORM plus views to encapsulate data. INSERT INTO invoice isn't all that bad if invoice is a view. Data operations are still encapsulated.

The second point is that I am a relative fan of loosely coupled systems. Loose coupling gets away from CAP theorem problems because each system is autonomous and can always defer interop with other applications until later. On the other hand your centralized RDBMS goes down and every app that needs it dies with it. How many apps we want entangled in that way is a business decision but there are tradeoffs on both sides.

That doesn't mean endorsing the NoSQL approach though. I maintain that you can't have "eventual consistency" in any meaningful, guaranteed way unless you always have "absolute local consistency" within some scope. Postgres-XC defines that scope as cluster-wide. In a loosely coupled system it could be each individual node. For this reason BASE doesn't work for anything that is important. It reduces to:

Best-guess Attempt at Semi-consistent Eventualities.

Lesson is: not only do you want to encapsulate your data, but anywhere that you want eventual consistency, you need to have local ACID compliance for some defined scope of local. This is a big issue that NoSQL db's have and that they cannot solve (due to CAP theorem issues). If only it were absent from the RDBMS world. I groan every time someone asks for "multi-master asynchronous replication."

Re: Why I Migrated Away From MongoDB

#212

Post by lmm » Thu, Sep 20, 2012, 10:22 AM UTC

einhverfr wrote: post #204

lmm wrote: post #197

Earlier quoted context omitted.

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting. I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on ou…

Obviously if your report makes assumptions about your data which aren't true then you might get invalid data out. I absolutely agree with having a single point through which writes to the data store must pass which enforces business-level constraints on the data. I just don't find SQL a convenient form to express those constraints (and my experience has been that any given business domain will have some constraints that are too complex to express in the SQL model, forcing you to resort to e.g. triggers - at which point the constraint is not integrated with the rest of your data model, it's just a piece of code that runs to check your inserts, which you could do equally well in any language); I'd rather do it in a "normal" programming language.

I see what you're getting at with reporting now, you're talking about doing actual calculations on the data? For mongodb I'd probably use its map-reduce API, at which point you're writing javascript and you can do anything, and performance should be fine. Though honestly other than performance I've never had a problem with just fetching the rows I need and doing whatever transformation/aggregation logic in e.g. python. SQL has never struck me as particularly typesafe or gaining much advantage from being close to the data; its sets-oriented way of thinking can be helpful in some places, but it's not the only language that can do that.

If you like SQL as a language for writing logic in I can see why a traditional database would appeal. But even then I feel like input validation, data storage and query processing should be separate layers (and I see some movement towards that with e.g. MySQL offering an interface to do key/value lookups directly). If SQL is a good way of doing transform/aggregation logic then it should stand alone, and I'd expect to see SQL-like query layers on top of key-value stores.

Re: Why I Migrated Away From MongoDB

#213

Post by AlisdairO » Tue, Sep 25, 2012, 7:10 PM UTC

einhverfr wrote: post #192

AlisdairO wrote: post #54

Earlier quoted context omitted.

Just in case you weren't already aware, the count(*) issue should be significantly improved for some queries in postgres 9.2, thanks to the addition of covering indexes.

There was discussion on pgsql-general a bit before the release. It appears that physically sequential scan of a table is about as fast as key-order sequential scan of a covering index. To speed up a lot, I think you'd need to be able to scan an index in physical order which is not currently supported. Now if you are using SSD's yeah, you could tweak the planner settings enough to cause it to make an index scan perhap…

ah, that's unfortunate :-(