Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

281–290 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#281

If you're currently using MongoDB in your stack and are finding yourselves outgrowing it or worried that an issue like this might pop up, you owe it to yourself to check out RethinkDB: https://rethinkdb.com/ It's quite possibly the best document store out right now. Many others in this thread have said good things about it, but give it a try and you'll see. Here's a technical comparison of RethinkDB and Mongo: https:…

I assume Rethinkdb would not have this bug, could anyone from the team confirm this is the case?

Re: MongoDB queries don’t always return all matching documents

#282
post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

So get them to use ToroDB.

Re: MongoDB queries don’t always return all matching documents

#283

MongoDB reminds me of an old saying that if you have a problem and you use a regex to solve it, you end up with two problems. I have personally used MongoDB in production two times for fairly busy and loaded projects, and both times I ended up to be the person that encouraged migrating away from MongoDB to a SQL based storage solution. Even at my current job there's still evidence that MongoDB was used for our produc…

I literally brought up the regex joke in a meeting yesterday. A data warehouse was built on top of Mongo, and I get to help clean up the mess.

Re: MongoDB queries don’t always return all matching documents

#284
post #223

Earlier quoted context omitted.

> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" f…

> I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" fields into a single "name" field with Mongo or a SQL DB is basically that the SQL DB has better tooling so it'll be cl…

> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously.

> Err no it isn't false. It is based on real-world experience, rather than "theory" about "what you can do". In the real world, people use ORMs and the ORM migration library, and then they find that the migration library doesn't do what they need it to do, so they have to use raw SQL. But then they run into the problem that because the ORM handles all of the generation of indexes and column names and so on, they have to figure out the correct naming of everything so that the ORM understands what you have done. This often results in failure because the ORM does not want you to tinker with the schema yourself.

It sounds like you're making a decent argument against manually modifying a database schema at the SQL level, behind the back of some ORM's which can't gracefully handle such surprises. But I don't think anyone is disagreeing with you about that.

> Which requires you to context switch from the code to the schema and piece together all of the relations based on foreign keys.

Many programmers can handle that complexity. Maybe the additional layer of indirection / complexity added by your ORM is the particular systems you deal with too hard for you to reason about?

Re: MongoDB queries don’t always return all matching documents

#285
post #104

A lot of Mongo DB bashing on HA. We use it and I love it. Of course we have a dataset suited perfectly for Mongo - large documents with little relational data. We paid $0 and quickly and easily configured a 3 node HA cluster that is easy to maintain and performs great. Remember, not all software needs to scale to millions of users so something affordable and easy to install, use, and maintain makes a lot of sense. Lo…

Ha ha, had to scroll all the way down to find a positive comment. It's actually a great paradigm; not every problem fits into a relational box.

Re: MongoDB queries don’t always return all matching documents

#286
post #225

Earlier quoted context omitted.

This list makes me want to cry a little. It rings too true. > Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). I literally just spent the better part of tonight AND yesterday evening dealing with one of these scripts. I had pulled down the production table to locally test the script (gross), but when I later ran it in the…

Oh god, I'm getting flashbacks. Because you can't just test it on one document and see if it works; you have no guarantee that all the documents will be identical. And if the migration script crashes halfway through... oh man.

> And if the migration script crashes halfway through... oh man.

Schema-issues and typing aside, I looked at MongoDB just long enough to find out there are no transactions, then ran away, quickly.

For a lot of tasks, I guess I would find MongoDB very useful, but lack of transactions is a complete deal breaker for me. Not having a real schema, referential integrity and all that makes them even more important, IMHO.

At work, I have had more than one quickly-hacked-together Perl script crash on me in the middle of a run. Having proper transactions has saved my butt repeatedly.

Re: MongoDB queries don’t always return all matching documents

#287
post #41

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Why on earth haven't I come across this information before? I spent a crazy amount of time researching frameworks before settling for Meteor, and never came across this.

Re: MongoDB queries don’t always return all matching documents

#288

This is exactly what I thought. You are a DBA. You are not a programmer. Whenever I read these threads I think "either these people are all DBAs or they have an extreme emotional attachment to a particular database". There it is - this is a thread full of DBAs rebelling against a database that takes away their power and responsibility.

I'm both, and I'm as equally at home writing C++ code as I am writing T-SQL or PL/SQL.

Frankly, getting to grips with relations and digging into functions made me a better programmer.

Re: MongoDB queries don’t always return all matching documents

#289

Earlier quoted context omitted.

> Mongo is a dumb, dead-end platform, but they know how important ease-of-use is. By “ease of use”, do you mean “ease of making something that seems to work” or “ease of making something that actually works”? I've never used a schema-free database, and ended up thinking to myself “I'm completely sure this database can't possibly contain garbage data”. Or do programmers simply not care about data integrity anymore?

The single biggest source of grief in our production database has been the one JSON field we used once to avoid adding another table. That goddamn thing has crashed the server so many times with invalid data, that I'm never using anything schemaless again. We recently migrated to a proper table and I'm thanking my lucky stars I finally got rid of that devil.

> I'm thanking my lucky stars I finally got rid of that devil.

All that I can say is congrats, man!

Re: MongoDB queries don’t always return all matching documents

#290
post #221

Earlier quoted context omitted.

> None of you have any idea what you are talking about even though you are supposed to be hackers. You heard it here first folks. None of us know anything.

> You heard it here first folks. None of us know anything. You've just taken the first step toward's web-scale enlightenment.

There is no spoon?
Post reply on HN