Live data from Hacker News

Think before you Mongo

blog.runnable.com

71–80 of 101 posts

Re: Think before you Mongo

#71

Earlier quoted context omitted.

That'd just be Postgres. When you need an schema you use a normal table, when you need arbitrary JSON data you use a JSON schema.

I don't think so -- I'm looking for a database that will actually enforce a JSON schema for you -- I don't think Postgres has any built-in support for JSON schematization. I could always do validation before inserting data, but that opens me up to error on my side which I'd like to avoid=)

> I don't think Postgres has any built-in support for JSON schematization.

Guess again. You can use JSON functions¹ in constraints:

  create table foo (
    bar jsonb,
    constraint bar_count_is_positive check (bar->>'count' is not null and (bar->>'count')::integer >= 0)
  );
etc

You could probably write a tool pretty easily to convert a JSON schema to Postgres constraints.

1. https://www.postgresql.org/docs/current/static/functions-jso...

Re: Think before you Mongo

#72
post #43

"We’ve lived and we’ve learned, and now we’re in the process of migrating from MongoDB to PostgreSQL." Why PostgreSQL instead of MySQL?

> Why PostgreSQL instead of MySQL?

Why, is there something inherently better in MySQL that makes their second choice also suboptimal? Genuine question, I have very limited knowledge on DBs.

Re: Think before you Mongo

#74
I am coming from an 18-year SQL admin/dev background, and am both an MCDBA and MongoDB certified DBA. I and my DEV team have been incorporating MongoDB into our stack for about 5 years now in multiple use-cases.

I am using MongoDB in my production environment for things like consuming incoming rates from different vendors, storing and serving pay stubs and client invoices to our web customers, controlling MSMQ message queues, archiving client emails for historical audits, etc. The key here is that they are document oriented entities, not normalized relational data.

What it comes down to is a willingness to be agnostic in selection of your database platform. Or more to the point, to let your use-cases drive the platform instead of the reverse. If you are going to develop a use-case that requires frequent partial updates, JOINs between multiple data structures, and traditional entity normalization, then a traditional RDBMS is appropriate. If your case allows for a denormalized mode of storage where all of the relevant information is contained within one document structure, and you can benefit from a fluid design, then MongoDB could be a good fit. If you need ephemeral key-value pair structures such as in session state caching, then something like Redis may be more in line with the requirements. They all have sweet spots that they fill well...

We all tend to have our preferred DB "hammer" to drive developmental "nails". What I propose is that we need to have an entire database toolbox from which to choose the right tool for each job.

Others have commented here about the need to thoughtfully plan before you write one line of code and/or choose your DB platform. I have to agree completely. You can map a typical relational use-case to MongoDB very easily to start with, but you do need to be able to enforce some level of control. MongoDB does this now with document validation.

https://docs.mongodb.com/manual/core/document-validation/

You also now have document level atomicity and transaction-like behavior as of MongoDB 3.2.

https://docs.mongodb.com/manual/core/write-operations-atomic...

MongoDB, like the rest of us, is constantly iterating and improving. If you have not looked at it lately and are basing your opinions on earlier versions, I would encourage you to take another look...

Just my humble opinion...

Re: Think before you Mongo

#75
post #22
post #12

Earlier quoted context omitted.

I am yet to fail into this NoSQL trend. Yes, SQL does have its issues, but I am yet to work on any project where the relational data model doesn't fit. Plus all the stuff that we can do at the SQL engine level, specially data validation, is just great for the type of stuff we develop.

NoSQL is driven more by scaling issues than anything else. Joins and strong consistency are awesome but run head first into the CAP theorem and other concerns like single node performance on a sharded cluster. There is also the fact that no programming language lets you deal with relational data sanely in code, so you have the well known impedance mismatch heaeache. All popular languages I've seen offer hierarchical…

The thing is, for the majority of deployments out there, developers don't even know what is the CAP theorem, nor do they need the theoretical performance NoSQL databases have over default configured RDBMS.

Just like all those big data deployments that can fit on an USB key and be processed by plain UNIX tools.

Re: Think before you Mongo

#76
The author freely admits they tried to do a relational schema in a non-relational database. Then proceeds to say it was the wrong choice of product, instead of design. This confuses me. I use MongoDB as my main database, and it works great for me. However, I did have to learn that it works differently, which meant writing my code differently.

My question is this, if you tried to use PostGres and designed your tables super super wide and embedded everything in large object data types and then it got to complicated to manage and figure out, would you consider that a failure of PostGres, or your failure as a developer/architect?

I will have to chock this up to a good laugh myself. Lastly, I've used relational databases for most of my career, and I'm very good at writing SQL. But after learning MongoDB and how to use it properly, there isn't a lot that I would want to go back to an RDBMS for. And with the future roadmap of MongoDB, I don't see that changing.

Re: Think before you Mongo

#77
post #9

like the yesterday's submission of "Why you should never use MongoDB (2013)" ( https://news.ycombinator.com/item?id=12290739 ), this is all just plain sad because totally unnecessary: the failure of hierarchical databases was obvious in 1960s and was the background of Edgar F. Codd's work on the relational model. unfortunately, this industry is dominated by cocky PFYs (of any age) convinced they don't need to study h…

Greybeard here.

I suggest you check out The End of an Architectural Era (It’s Time for a Complete Rewrite)[1].

The "failure" of non-relational databases was for specific application domains.

[1] http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf

Re: Think before you Mongo

#78
post #69

Earlier quoted context omitted.

You have to separate between "Mongo the database" and "Mongo as it's used by companies"; the latter causes far more problems than the former. I last used MongoDB seriously in 2012-2015. We had myriad operations problems including inconsistent indexing across shards (where some shards had an index created and others didn't, it was baffling), issues with the balancer not moving chunks properly, and more. Also it's just…

I agree with your points about typical database usage. Could it be that some of the people who reach for a database don't in fact know what a particular database implementation (like Mongo, or MySQL, or Redis) actually does, and they're just looking for a black-box that holds data at rest and occasionally gives it back out?

Yeah, maybe. But there's a bigger point here and that's that software engineering is starting to become more of a "there's a way we do things" craft profession (like medicine, law, or architecture) vs. a "everything is from first principles all the time" endeavor. There's just too much stuff to know. You can't expect people to have deep experience with more than, say, 2-3 databases every decade.

So we have to rely to some extent on the experience of others and our own intuitions / less than perfect inferences.

Re: Think before you Mongo

#79

Earlier quoted context omitted.

You have to separate between "Mongo the database" and "Mongo as it's used by companies"; the latter causes far more problems than the former. I last used MongoDB seriously in 2012-2015. We had myriad operations problems including inconsistent indexing across shards (where some shards had an index created and others didn't, it was baffling), issues with the balancer not moving chunks properly, and more. Also it's just…

I would say the latter and the former are inexorably intertwined; i.e. Mongo causes problems for companies because it does things you would not expect a database to do, like your example of phantom indexes...and pretty much all bets being off when you try to leverage sharding... the one thing it was supposed to be able to do to scale...

Couldn't agree more.

We had more problems with sharding over the years than you could imagine...the distributed locking mechanism didn't work a lot of the time, the balancer didn't work, weird consistency issues between the config servers, configuration that didn't get replicated across all shards, stupid shard key selection (admittedly our fault but there really should be better guidance on this topic), etc.

Re: Think before you Mongo

#80

I agree... but at this point, it's tough to see with all the ink that's been spilled on these issues for years how you could think anything else. Maybe I read HN too much, but the manifold problems with MongoDB have been widely publicized for the past 6 years... it seems pretty close to conventional wisdom that you're going to have those problems if you decide to use Mongo.

Because Mongo is wrongly marketed as "all-in-one" database. Instead it should be marketed as "good database for dynamic data that requires complex filtering, also we have very good drivers for many languages" because that's the only thing it's good at.

Yeah. I remember thinking a few years ago that it tries to be three totally different products at the same time: a straight NoSQL k-v store like S3, a SQL-like OLTP relational thing ("rows are sorta like documents, right") and an offline analytics store with the map/reduce functionality.

It's like they wanted to be all three but couldn't quite commit to one, and the end result is the proverbial "tankicopter" that's neither as strong as a tank, nor as maneuverable as a helicopter.

Post reply on HN