Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

221–230 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#221

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> None of you have any idea what you are talking about even though you are supposed to be hackers.

You heard it here first folks. None of us know anything.

Re: MongoDB queries don’t always return all matching documents

#222
post #101
post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Because postgres doesn't focus on the first-five-minutes experience the way Mongo does. Even the name is hard to say. Mongo is a dumb, dead-end platform, but they know how important ease-of-use is.

What they neglect to mention is not that you don't need a DBA, that you are the DBA yourself. With all the responsibilities that go along with that role. Who's getting paged at 3am now...?

Re: MongoDB queries don’t always return all matching documents

#223

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql

I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice.

The difference between, eg, merging your "first_name" and "last_name" fields into a single "name" field with Mongo or a SQL DB is basically that the SQL DB has better tooling so it'll be cleaner, faster, and more concise.

> Now in your migration script you have to hack SQL and language-based ORM commands

Also false. Nothing requires you to use the ORM commands if they don't make your life easier; literally last week I wrote a migration for some ORM based code using pure vanilla SQL because it was more appropriate. If you know what your ORM is doing, it's trivial. And if you don't, well, you have bigger problems.

> Actually they can be just as explicit and documented.

With a relational DB: Check the table schema.

With mongo: Dig through the app code and try and reverse engineer what fields the code is expecting. (Or pull up a few documents and see what fields actually exist, then ponder how you know whether you're looking at document that uses the latest schema, or maybe an outdated or dead document using an outdated schema. Also, how do you know what optional fields might be missing? Or what possible values the currently null fields might have? Good luck with that.)

> Mongo even has schemas now I think.

No. Mongo still lists flexible schemas as a feature, ie, they do not enforce any specific schema.

> SQL constraints are very limited and literally every application has additional constrains

Yeah, but what relational DBs are very good at is enforcing foreign key constraints. If your data is relational at all (and let's be real, very, very little world data isn't at least somewhat relational), you'll need it, and Mongo doesn't have it. (Well, Mongo doesn't even have foreign keys, but it lacks the equivalent feature for dealing with denormalized data too.) Managing to set some non-numeric characters in your phone number field when some of your app code contains a hard assumption it will be numeric is bad, yeah. But when you start screwing around with relations, foreign keys, failing to properly propagate changes to every copy of a denormalized data structure...oh man, you can spend days trying to untangle that mess. And it's a class of error that relational DBs don't have unless you misuse them badly.

In short, you seem to be suggesting that as long as you're blindly using an ORM, Mongo is almost as good and flexible as Postgres. Which...sure, okay? I guess?

Re: MongoDB queries don’t always return all matching documents

#225
post #124

Earlier quoted context omitted.

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

This list makes me want to cry a little. It rings too true. > Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). I literally just spent the better part of tonight AND yesterday evening dealing with one of these scripts. I had pulled down the production table to locally test the script (gross), but when I later ran it in the…

Oh god, I'm getting flashbacks.

Because you can't just test it on one document and see if it works; you have no guarantee that all the documents will be identical. And if the migration script crashes halfway through... oh man.

Re: MongoDB queries don’t always return all matching documents

#226
post #9

Earlier quoted context omitted.

> don't use mongo, it's just hype I'm kind of curious as to where this hype is. I've almost never heard anybody say anything positive about mongodb. All I ever see is people saying it's terrible / hilarious for various reasons.

The hatred for MongoDB mostly comes from the PostgreSQL supporters camp. The rest of us are just using whatever tool makes sense. MongoDB is the fastest database I've ever used. The easiest to get running, stable and scaled out. The best documentation by far and has excellent integration e.g. Spark, Hadoop. If you are doing Big Data it's a great tool in the arsenal.

RethinkDb does all of this better, and with good engineering and API design behind it.

Re: MongoDB queries don’t always return all matching documents

#227
post #53
post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Because Postgres doesn't have any clustering / HA features. So out of the box it doesn't scale well.

It HAS them, just not built-in tooling to make using them easy. 2ndQuadrant's repmgr gets you partway there, I'm really hoping to see them revamp it now that pg_rewind is a thing to make restoring a failed master less of a pain in the butt (this is literally the only reason I don't bother with HA right now, it's usually much easier for me to get the DB back online or restore from a barman backup than deal with replication).

Re: MongoDB queries don’t always return all matching documents

#228

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

NodeJS + MongoDB is this generation's Laurel and Hardy stack ("look at this mess you got me into"). Last generation's was PHP + MySQL.

Actually PHP and MySQL is a pretty respectable stack today if you use it right.

Re: MongoDB queries don’t always return all matching documents

#229
post #126

Everytime I hear arguments for going back to relational databases, I remember all the scalability problems I lived through for 15 years in relational hell before switching to Mongo. The thing about relational databases is that they do everything for you. You just lay the schema out (with ancient E-R tools maybe) load your relational data, write the queries, indexes, that's it. The problem was scalability, or any toug…

> And the SQL-way, with PK constraints and things like triggers, just made matters worse by claiming the database was actually responsible for maintaining data consistency.

I...just...I can't.

Your database should ALWAYS be responsible for maintaining consistency. Your application will likely be dead in a couple years, and if it isn't then you are going to end up having something else interfacing with the database at some point, guaranteed. The only sane place to guarantee consistency is in the database itself, otherwise every time you change constraints you are going to be updating every integration you have and hoping you didn't miss something.

My lord, what has come over developers?

Re: MongoDB queries don’t always return all matching documents

#230

Earlier quoted context omitted.

> Mongo is a dumb, dead-end platform, but they know how important ease-of-use is. By “ease of use”, do you mean “ease of making something that seems to work” or “ease of making something that actually works”? I've never used a schema-free database, and ended up thinking to myself “I'm completely sure this database can't possibly contain garbage data”. Or do programmers simply not care about data integrity anymore?

Well, I have used one and had that assertion. Ease of use means making something that works. It seems we're condoning lack of knowledge or experience with programming. If you've never used databases in your life, whether you're using SQL orNoSQL you'll likely end up with rubbish data. In the SQL world it could be that you're storing time in the wrong format, concatenating long fields, or not normalising when you shou…

> Well, I have used one and had that assertion. Ease of use means making something that works.

For me, it means, under no circumstance, no interleaving of transactions or scheduling of commands, nothing, nichts, nada, can the database be in a state where a business rule is violated. If I need to worry what silly intermediate transaction state can be observed from another transaction, or if I need to worry whether a master record can be deleted without cascade-deleting everything that references it, then the DBMS has failed me.

> not normalising when you should or the other way around.

I've never seen a situation where anything less than 3NF (actually, ideally, at least EKNF) is acceptable.

Post reply on HN