Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

171–180 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#171
post #90

Earlier quoted context omitted.

Are there NoSQL alternatives that have a worse track record than Mongo?

Cassandra, HBase etc had checkered pasts with plenty of their own data loss and inconsistency bugs. Now they are considered two of the most rock solid NoSQL databases. The hatred towards MongoDB really is pretty irrational given just how popular the database is.

It is anything but irrational.

https://aphyr.com/posts/322-jepsen-mongodb-stale-reads

If it isn't already fixed, they need to fix it. I can't vouch for HBase, but Cassandra is stable and reliable now. Cassandra is also easier to scale.

Re: MongoDB queries don’t always return all matching documents

#172
post #103

CouchDB is simple and reliable. You can understand it from day one. I can't imagine why it isn't being used.

I really want an excuse to build something with CouchDB and PouchDB ( https://pouchdb.com/ ). Can you expand on your experiences with it?

I'm not the original poster, but I can give you some of my limited experience with CouchDB from an application I inherited. The original idea for the project still seems like a good idea to me. Basically they wanted to record events that came into the system and store them in a write only ledger. Then they wanted to version every change so that you have an audit trail. Finally they wanted to be able to create views of that ledger to create the kind of data that they would work with on a day to day basis. For this, CouchDB seems like a perfect fit.

Unfortunately, it didn't work out as well as one might hope because the people who implemented the idea didn't seem to be able to resist using the DB the way they would use a relational db. Instead of maintaining the concept of a write only ledger, they started to use it as a data store for things that were ephemeral. Also, instead of replicating the db, using a view to create a new db that was optimal for certain queries, they wrote a huge number of views in the main db. Finally they organised the views by relation rather than by use, so you would have 60-80 views in the same design document that would have to be reindexed if one of them changed.

The result was something with very poor performance and where the storage for the indexes was more than an order of magnitude more than the storage for the documents themselves.

CouchDB is also not super speedy at the best of times. There is a lot of latency involved in serializing the documents and farming them out to view servers, etc. So it takes a good 10 minutes to process a million documents, but you will find that your CPU is chugging along at 30-40% utilisation.

Having said all that, one of the things I want to try (but have only done some preliminary trials with) is to keep the concept of the write only ledger, but to replicate the db into several views of the data (some with severely restricted content). Then instead of building something like a rails application to farm out the data, make "couch applications" where you serve the HTML and JS directly from attachments on documents in the DB. In fact, I've written a React application to allow users to interact with portions of the data and it was quite simple. Then you can write a really small coordinating application to allow users to navigate to the parts of the system (really single page apps) that they want to use.

Again, the nice thing about this is that you have a write only data store with versioning and the ability to audit history. You have views that allow you to interact with a small subset of the overall data. You can easily write single page applications where deployment is as easy as pushing a document to the DB. Replication is relatively cheap and you can move expensive view creation to restricted versions of the DB. You can stick the whole thing behind a load balancer and scale it as cheaply as setting up a new replication (again just another document in your DB).

But, I will warn you. Don't use it like you would a relational DB, or else you will be in for a world of hurt. Especially you will see comments in this thread about migrations. If you are migrating your data, by definition you do not have a write-only-with-versioning application. Your application will have to deal with multiple versions of data or else you will not have the ability to audit history. If you do not care about this, then possibly there are better solutions than this.

Re: MongoDB queries don’t always return all matching documents

#173
post #52

Earlier quoted context omitted.

Yes - with Apollo/GraphQL (currently available as a technical preview): http://docs.apollostack.com/apollo-client/meteor.html I recommend you check out the Apollo Meteor Starter Kit: https://github.com/apollostack/meteor-starter-kit

Noticed how I referenced 2 proper RDBMS in my question? Then how you proceeded to introduce another flavour-of-the-month

Did you look at the Apollo link? it allows you to use any db you choose. some of the starter apps use mongo,postgres, rest api. it's quite cool.

Re: MongoDB queries don’t always return all matching documents

#174
post #41

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

This will be highly subjective but you need to get over the "postgresql has the longest feature list so why don't you use it". The last startup I have been involved with tried to use PostgreSQL and needed to move to MySQL (yeah, well) because commercial support was both more expensive and less useful than what we were able to get for MySQL. Perhaps today it's different.

While I no longer use PostgreSQL much, every time I need to touch it seems rather developer unfriendly, just last month I found MySQL, heck even SQLite supports triggers with code inlined into the trigger body but PostgreSQL mandates writing a separate function for the trigger. And, of course, it needs to be in plpgsql because reasons. The most trivial "let's calculate another column" becomes a complicated nightmare.

So then if you don't want to use PostgreSQL what then? The answer now is MySQL, again, because 5.7 has JSON.

And mind you, I have grown to dislike MongoDB slowly over the years as new types of queries have appeared and it's a complete mess by now. There was an excellent article on this posted on Linkedin of all places this March https://www.linkedin.com/pulse/mongodb-frankenstein-monster-...

It's really interesting how MySQL is the most usable and most supported database by now...

Re: MongoDB queries don’t always return all matching documents

#175
post #101
post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Because postgres doesn't focus on the first-five-minutes experience the way Mongo does. Even the name is hard to say. Mongo is a dumb, dead-end platform, but they know how important ease-of-use is.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps.

Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Re: MongoDB queries don’t always return all matching documents

#177

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

Is having seen something used some way a leading indicator of it being a good idea to have used that thing that way? Because I've seen Excel used as database with all kinds of macros and VBA scripts bolted-on/embedded to provide the workbook various shapes of stored-procedure and query capability... but, while sorta impressive in a "Holy crap, lol wut?" kind of way, I'm not sure any instance I observed of uses like t…

Did they make the company a lot more money than they cost? If so they were probably a good idea. Not all code needs to be "pretty" to serve a purpose. I've seen some pretty epic hacks that I know generated hundreds of thousands of dollars of new revenue.

Re: MongoDB queries don’t always return all matching documents

#178
post #124

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

Schema on read, as opposed to schema on write, as it says in the excellent "Designing Data Intensive Applications" book ... nightmarish to deal with.

Re: MongoDB queries don’t always return all matching documents

#179

Earlier quoted context omitted.

Cassandra, HBase etc had checkered pasts with plenty of their own data loss and inconsistency bugs. Now they are considered two of the most rock solid NoSQL databases. The hatred towards MongoDB really is pretty irrational given just how popular the database is.

It is anything but irrational. https://aphyr.com/posts/322-jepsen-mongodb-stale-reads If it isn't already fixed, they need to fix it. I can't vouch for HBase, but Cassandra is stable and reliable now. Cassandra is also easier to scale.

FWIW, I don't think Cassandra is particularly any better today semantically than it used to be. Merge conflicts are still at the cell level than the row level, and wall-clock time is still the way that LWW resolution is determined. It let's you mix strongly consistent and eventually consistent data together, which makes no sense.

But the difference is that Cassandra is reliably "broken" in those ways, and as a result there are ways of using it which don't lean heavily on those weaknesses. Such as writing only immutable data or isolating all data that will be used in paxos transactions into their own column families by convention, etc.

Cassandra more or less behaves exactly as it claims that it does. So you can do a somewhat thorough investigation of its system semantics and know what you can rely on and what you can't. MongoDB doesn't even uphold the system semantics it claims that it has, so it's just broken in weird and esoteric ways that you discover mostly by accident.

Post reply on HN