Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

Post reply on HN
Print view View original

311–320 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#311

Post by jeltz »

snaky wrote: post #298
1879325235 wrote: post #204

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> SQL constraints are very limited Oh really? What exactly cannot be expressed in PostgreSQL check constraint, given the fact you can write it in any programming language you prefer?

Arbitrary constraints which look at multiple rows require serializable isolation to work under concurrency, and most people do not run their databases at serializable. But you get pretty far with single row check constraints and exclusion constraints (exclusion constraints can be used for checking against range overlap).

Re: MongoDB queries don’t always return all matching documents

#312

Post by te_chris »

vidarh wrote: post #305
eloff wrote: post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Here's one: You're dealing with a torrent of incoming semi-unstructured data, where losing a good chunk of it is minor nuisance because you only need a decent sample, from which you extract data. In those kind of scenarios, making it easy to work on the code can often be far more important than reliability. I have a project like that now. I'd love to use Postgres, and probably will eventually once things "settle down…

That's a totally valid reason to use Mongo, now if only they'd market themselves that way.

Re: MongoDB queries don’t always return all matching documents

#313

Post by creshal »

spriggan3 wrote: post #76
eloff wrote: post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

> Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that

So use an ORM that understands Postgres' JSON columns. Don't need to write a single SQL statement, automagic migrations, no explicit schema (unless you make one), no constraints (unless you add them).

It works great, we did a rather large project last year using Django's ORM and postgres where we didn't know the final data schema until months after launch.

Re: MongoDB queries don’t always return all matching documents

#314

Post by danbmil99 »

Oh, the fud of it.

The behavior is well documented here https://jira.mongodb.org/browse/SERVER-14766

and in the linked issues. Seasoned users of mongodb know to structure their queries to avoid depending on a cursor if the collection may be concurrently updated by another process.

The usual pattern is to re-query the db in cases where your cursor may have gone stale. This tends to be habit due to the 10-minute cursor timeout default.

MongoDB may not be perfect, but like any tool, if you know its limitations it can be extremely useful, and it certainly is way more approachable for programmers who do not have the luxury of learning all the voodoo and lore that surrounds SQL-based relational DB's.

Look for some rational discussion at the bottom of this mongo hatefest!

Re: MongoDB queries don’t always return all matching documents

#315

Post by smanuel »

lossolo wrote: post #4

I've just migrated one project from mongo to postgresql and i advise you to do the same. It was my mistake to use mongo, after I've found memory leak in cursors first day I've used the db which I've reported and they fixed it. It was 2015.. If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db…

> If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db do it for you.

So... you are not against MongoDB but against NoSQL in general? I've used MongoDB and I've never ended up with lots of joins in my code. But I guess it all depends on the use case and how you've structured your data. Document databases are not a silver bullet.

Re: MongoDB queries don’t always return all matching documents

#316

Post by jeltz »

blackoil wrote: post #276
manigandham wrote: post #175

Earlier quoted context omitted.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps. Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Postgres members should listen this and have a simple getting started guide for osx, Windows, Linux. I tried brew install postgresql. There was no single place which tells me how to start server, access command line, create db etc.

Getting started with PostgreSQL on Linux is actually trivial. What is annoying though that there are lots of guides which talk about editing pg_hba.conf which is not necessary for the simplest setup. The default pg_hba.conf is good in most distros.

Re: MongoDB queries don’t always return all matching documents

#317

Post by Someone »

ptman wrote: post #252
sandGorgon wrote: post #98

Earlier quoted context omitted.

in this case, please use Postgresql jsonb data tables. Just as powerful as mongo... but with the stability and guarantees of postgres

It has been pointed out before that json(b) has problems with indexing. IIRC the cost estimates of indexes on JSON data are static , and therefore very rarely accurate. I'm terribly sorry but couldn't find a reference with 5min of searching. I still like postgres over mongo

http://postgresql.nabble.com/working-around-JSONB-s-lack-of-...

Of course, that doesn't mean indexes never help. See for example http://blog.2ndquadrant.com/jsonb-type-performance-postgresq....

I guess the workaround would be creating indexes on computed columns that query from the json data, together with changing one's queries to use that computed field. For example, with a json column storing names in various places, a computed column could collect all of them in an array. An index on that computed column will have good statistics.

Bottom-line: if you want your queries to run fast, you will have to tell your store what kind of data you have and what kind of queries you will run. Otherwise, there's little the store can do.

Having a traditional database with various constraints is a way to give that information. With json columns, you may have to do it in another way (for now).

Re: MongoDB queries don’t always return all matching documents

#318

Post by jeltz »

bigiain wrote: post #185
PeCaN wrote: post #50

Earlier quoted context omitted.

MongoDB: Because /dev/null doesn't support sharding.

nc -vv -l 27017 > /dev/null That and some round robin dns entries should be good enough for prod use, yeah? ('course it's not webscale ready till it's deploayble as a docker container which contains critical nodejs code with a dependancy on leftpad.js...)

If you use ncat you can even get TLS support.

Re: MongoDB queries don’t always return all matching documents

#319

Post by PeterisP »

joncrocks wrote: post #279
gaius wrote: post #257

Earlier quoted context omitted.

Oh, that doesn't happen much, because serious organizations have dozens of apps connected to a DB and no-one would be crazy enough to big-bang change them all at once to use a different schema. Adding columns and tables tho' is easy, populating new tables from existing, replacing tables with views, no problem.

I'd argue the inverse. Serious organisations shouldn't have lots of different apps reading from each other's databases, for the reason you suggest above. It makes changes difficult and the application landscape fragile. It also encourages not having well-documented interdependencies between your apps (and consequently no way of knowing what the impact of any given change to an app will have). It's not uncommon howeve…

In such a scenario, none of the apps "own" the data - the DB is a primary source of truth and consistency is enforced there instead of relying that the code in the many apps will match.

Application logic can undergo active changes and refactoring without changes to the data permanent storage, indeed, a 1-to-1 correspondence between DB and application data structures isn't expected. It's just as with backwards compatibility with file formats, changing the application version shouldn't require incompatible in-place changes to the backing data persistence layer - you can need to add some new features, but the old version of the app should work fine with the updated DB.

Re: MongoDB queries don’t always return all matching documents

#320

Post by badthingfactory »

im_down_w_otp wrote: post #16

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

It's pretty easy to use... until you have to normalize data and query across one or two joins. I've been forced to build with mongo for the past few months (still not sure why) and I can't think of a single valid use-case for this rubbish.

If you need denormalized/distributed caching, Redis does a good job. If you need to store some unstructured json blobs, postgres and now sql server 2016 can do that. If you need reliable syncing for offline capable apps, you probably want CouchDB. If you need real time, use Rethink Obviously, relational data belongs in a relational database.

I think the problem is that all of these databases do one or two things really well. Mongo tries to do all of these things, and does so very poorly.

Post reply on HN