Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

71–80 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#71
post #22

While I love to hate on MongoDB as much as the next guy, this behavior is consistent with read-committed isolation. You'd have to be using Serializable isolation in an RDBMS to avoid this anomaly.

In postgres (and a fair number of other databases) you'll not see that anomaly, even with read committed. Usually you'll want to have stricter semantics for an individual query, than for the whole transaction.

Re: MongoDB queries don’t always return all matching documents

#72

Earlier quoted context omitted.

We start our apps with mongo, and design them with a migration plan to postgres. We've found it's very easy to rapidly develop the application with mongo due to it's flexibility. Once we understand where or app is headed and what our relationships actually are, we pretty much pull the plug out of mongo and stick it in postgres. If you build a reasonably intelligent query wrapper it's fairly effortless. That being sai…

This feels like you lack an architect who can see the bigger picture of your applications. I don't mean that insultingly, but with experience you tend not to need that second system syndrome. Do you find there's less rewriting over time as you become more experienced? Or is it just the kind of projects you work on?

That's not what "second system syndrome" means.

Re: MongoDB queries don’t always return all matching documents

#73
post #52

Earlier quoted context omitted.

Yes - with Apollo/GraphQL (currently available as a technical preview): http://docs.apollostack.com/apollo-client/meteor.html I recommend you check out the Apollo Meteor Starter Kit: https://github.com/apollostack/meteor-starter-kit

Noticed how I referenced 2 proper RDBMS in my question? Then how you proceeded to introduce another flavour-of-the-month

Noticed how I referenced 1 proper RDBMS in my question?

Fixed that for ya.

Yes yes I know, HN is not Reddit. But there's still something fishy about declarations of what is/isn't a "proper" RDBMS.

Re: MongoDB queries don’t always return all matching documents

#74
post #22

While I love to hate on MongoDB as much as the next guy, this behavior is consistent with read-committed isolation. You'd have to be using Serializable isolation in an RDBMS to avoid this anomaly.

I think this is incorrect, but it's not as simple as the other replies are making it out to be.

Under read-committed isolation, within a single operation, you must not be able to see inconsistent data. So if you do "SELECT " on a table while rows are being updated, you're guaranteed to always see either the old value or the new value. But if you do two separate statements, "SELECT WHERE value='new'" and "SELECT WHERE value='old'" in the same transaction, you may not see the row because its value could have changed. Serializable isolation prevents this case, typically by holding locks until the transaction commits.

It gets messy because the ANSI SQL isolation levels are of course defined in terms of SQL statements, which don't map perfectly to the operations that a MongoDB client can do. Mongo apparently treats an "index scan" as a sequence of many individual operations, not as a single read. So you could argue that it technically obeys read-committed isolation, but it definitely violates the spirit.

Re: MongoDB queries don’t always return all matching documents

#75

Earlier quoted context omitted.

I was always a big fan of calling MongoDB the "Snapchat for databases". I think people even had some stickers printed for it..

Hey! Snapchat is pretty useful for seeing snapshots into the not completely dull parts of my friends lives. It has its potential flaws in being able to save the images/videos but please don't do it the disservice of likening it to MongoDB.

Oops - corrected, I meant "Snapchat for Databases" rather than "the Snapchat of Databases". Also, semi-OT: your reply was hilarious.

Re: MongoDB queries don’t always return all matching documents

#76
post #41

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it?

Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas.

In the context of analytics, it might make sense, I'm not a big data analyst, but I've seen MongoDB used to centralize logs.

Re: MongoDB queries don’t always return all matching documents

#77
post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Postgres wasn't always a great document store...there was definitely a time period where if you wanted to take a document-oriented approach to data modeling, MongoDB was a good way to go. JSONB was only added in the last minor version of Postgres, and while the JSON and HSTORE types were available, it didn't give you quite the same speed. Now that JSONB is a thing, I think the two databases are more comparable as a d…

Is that actually true? I mean the part about Postgres might be, I don't know. But was there a time when MongoDB was a good way to go?

Was there ever a time when it actually worked consistently well at something that was database shaped? Because I started dealing with it in ~2010 I think, and it wasn't a suitable database for anything other than toy projects or throw-away data back then, and while it's many versions newer, it still appears to be pretty fast and loose with its supposed system guarantees.

There was a point when they raised $100+ Million in funding that I thought they'd take that money and actually build a database. At least as recently as last Summer that wasn't a reality yet.

Re: MongoDB queries don’t always return all matching documents

#78
post #66

I believe this is solved by Mongo's "snapshot" method on cursors: https://docs.mongodb.com/v3.0/faq/developers/#faq-developers...

If I understand correctly, this method says "only scan the built in _id index, not any other index". Which means that you will not hit this index-specific bad behavior, but also that you won't get the performance characteristics of using an index.

Re: MongoDB queries don’t always return all matching documents

#79
post #21

TL;DR During updates, Mongo moves a record from one position in the index to another position. It does this in-place without acquiring a lock. Thus during a read query, the index scan can miss the record being updated, even if the record matched the query before the update began.

mongo developer 1 Man, it's just taking too damn long to run this update. What should we do? mongo developer 2 Uh....remove all the locks? mongo developer 1 Oh, yeah, that makes sense, let's do that. mongo developer 3 Hey, what are you guys up to? mongo developer 2 Making it web-scale! mongo developer 3 Good job, keep it up!

Seriously. I was looking at DB usage statistics recently and was appalled MongoDB is still so popular. I thought it was done, nail in the coffin, when https://www.youtube.com/watch?v=b2F-DItXtZs came out 6 years ago, I haven't followed it much since then apart from the occasional post like this whose content is just "you thought it was bad already? haha it's worse."

Re: MongoDB queries don’t always return all matching documents

#80
post #4

I've just migrated one project from mongo to postgresql and i advise you to do the same. It was my mistake to use mongo, after I've found memory leak in cursors first day I've used the db which I've reported and they fixed it. It was 2015.. If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db…

ArangoDB in contrast to MongoDB is a ACID document store also with graphs.
Post reply on HN