Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

Post reply on HN
Print view View original

321–330 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#321

Post by lars_francke »

danbmil99 wrote: post #314

Oh, the fud of it. The behavior is well documented here https://jira.mongodb.org/browse/SERVER-14766 and in the linked issues. Seasoned users of mongodb know to structure their queries to avoid depending on a cursor if the collection may be concurrently updated by another process. The usual pattern is to re-query the db in cases where your cursor may have gone stale. This tends to be habit due to the 10-minute cursor…

I wouldn't call a JIRA ticket good documentation.

While I agree that it's good to know the limitations of the tools you chose those limitations should be clearly spelled out in the documentation.

I don't think most programmers have the luxury of learning all the voodoo and lore that surrounds MongoDB from JIRA tickets and blog posts.

Re: MongoDB queries don’t always return all matching documents

#322

Post by PeterisP »

hk__2 wrote: post #95
spriggan3 wrote: post #76

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

It’s useful for prototyping. When you don’t know which schema you’ll end up using having an *SQL database is tedious because you have to do migrations every time you change the schema. Once you’re done prototyping you can switch to a better alternative.

In that case you can start with postgres and stuff your documents in a single json column, accessing it just as you would have in mongo while you're prototyping and don't care about speed and indexing, and when you're done, you can just change that table to a more proper structure without changing databases.

Re: MongoDB queries don’t always return all matching documents

#323

Post by virmundi »

hk__2 wrote: post #95
spriggan3 wrote: post #76

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

It’s useful for prototyping. When you don’t know which schema you’ll end up using having an *SQL database is tedious because you have to do migrations every time you change the schema. Once you’re done prototyping you can switch to a better alternative.

I guess this is a question of how usefuly, deployable the prototype shoukd be. Why not just have an in memory object cache, literally a hashmap, for your dal? If you're composing app level code, you don't need to know what the backend does to your data. You could even create a simple method to populate the data at app boot in your in the dev profile. When you figure out the storage requirements and finalized model, build your db.

This would save you time on picking the db, schema changes or even migration changes in Mongo. You don't have to worry about bad documents from an earlier app revision.

Re: MongoDB queries don’t always return all matching documents

#324

Post by orf »

1879325238 wrote: post #292
orf wrote: post #261

Earlier quoted context omitted.

Tl:dr - you used a crappy ORM and now think all ORM libraries sick. So... use MongoDB... which is always accesses through an ORM... All your comments come across as confused mate. Enforcing arbitrary constraints belongs in the business logic of your app, but dealing with consistency issues does not. Postgres handles row constraints just fine, you can enforce any arbitrary logic (i.e colX != colY and its not the first…

Mongodb isn't always accessed through an ORM. I have no idea what you are talking about. > Enforcing arbitrary constraints belongs in the business logic of your app Oh so you have some constraints enforced in your database, like the length of a strength can't be greater than 5, yet others you consider arbitrary just because they are not in SQLs list of constraints? How on earth did you define arbitrary otherwise? Any…

You're replies in the comments are rude and borderline fanatical in tone. There are two classes of constraints: one being 'this string shouldn't be longer/shorter than x' or 'this value should only be between 1 and 10'. The other kind is 'on the first Tuesday of every month only during a full moon can this record be deleted, and only by a user with permission X'.

The first can easily be modelled in SQL, the second should be modelled in your app. The first are not arbitrary, the second are. Not hard to comprehend mate. Just because SQL can't model all constraints doesn't mean you should throw it out the window and reach for mongo.

Comments like 'none of you understand this because youre not programmers and therefore know nothing' is incredibly insulting. I'm of the opinion that you know nothing, and if you're happy knowing nothing with fellow mongodb users then so be it. Just keep your nonsense away from me thanks.

Re: MongoDB queries don’t always return all matching documents

#325

Post by tremon »

jeltz wrote: post #311
snaky wrote: post #298

Earlier quoted context omitted.

> SQL constraints are very limited Oh really? What exactly cannot be expressed in PostgreSQL check constraint, given the fact you can write it in any programming language you prefer?

Arbitrary constraints which look at multiple rows require serializable isolation to work under concurrency, and most people do not run their databases at serializable. But you get pretty far with single row check constraints and exclusion constraints (exclusion constraints can be used for checking against range overlap).

Can you give a use-case for such a constraint that can not be avoided by a better data organization? I'm not sure why you would deliberately design your data with constraints based on other row contents.

Also: how do you enforce such constraints with MongoDB exactly? If the answer is "do it in the application", then your answer applies to relational databases too.

Re: MongoDB queries don’t always return all matching documents

#326

Post by PeterisP »

collyw wrote: post #254
gaius wrote: post #222

Earlier quoted context omitted.

What they neglect to mention is not that you don't need a DBA, that you are the DBA yourself. With all the responsibilities that go along with that role. Who's getting paged at 3am now...?

If thats really a problem host your database on a cloud platform and let those guys do the job of a DBA. Works for me at present. I am aware its not going to be a solution for everyone, though it still sounds a lot better than being your own Mongo DBA.

Cloud service doesn't really replace DBA.

Yes, it will help you to cover cases like where the server phyically explodes, but that's basically irrelevant, most problems where you need a DBA are caused either by data corruption caused by application code or developer, or performance issues caused by DB structure - in those cases the cloud platform won't do anything for you, they just host the server. They can restore backups, do monitoring and tune the server, not your particular app/db structure - but all the big problems are there.

Re: MongoDB queries don’t always return all matching documents

#327

Post by zamalek »

spriggan3 wrote: post #76
eloff wrote: post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

> some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that

Use the right tool for the job, right? Admittedly something like MongoDB could be the right tool for the job (examples around here include RethinkDB and CouchDB). MongoDB, however, is like a hammer with no head.

Re: MongoDB queries don’t always return all matching documents

#328

Post by tremon »

chris_wot wrote: post #290
DoofusOfDeath wrote: post #280

Earlier quoted context omitted.

> You heard it here first folks. None of us know anything. You've just taken the first step toward's web-scale enlightenment.

There is no spoon?

There can be only cloud.

Also, the cloud is Turing complete and has infinite tape.

Re: MongoDB queries don’t always return all matching documents

#329

Post by jeltz »

shruubi wrote: post #61

Seriously, who looks at MongoDB and thinks "this is a sane way of doing things"? To be fair, I've never been much of a fan of the whole NoSQL solution, so I may be biased, but what real benefits do you gain from using NoSQL over anything else?

Benefits of using MongoDB? Nothing. There are on the other hand other NoSQL systems which offer real benefits. Like Cassandra which gives reliable distributed database with not too much effort.

Re: MongoDB queries don’t always return all matching documents

#330

Post by orf »

1879325230 wrote: post #303
1879325238 wrote: post #292

Earlier quoted context omitted.

Mongodb isn't always accessed through an ORM. I have no idea what you are talking about. > Enforcing arbitrary constraints belongs in the business logic of your app Oh so you have some constraints enforced in your database, like the length of a strength can't be greater than 5, yet others you consider arbitrary just because they are not in SQLs list of constraints? How on earth did you define arbitrary otherwise? Any…

What do you mean what do I mean? I just gave you an example. You don't seem to understand that "consistency" in a database, is simply defined within the realms of the database itself. It isn't a universal concept of "consistency". It refers to the fact that the database will enforce certain constraints and checks. You can even denormalise your data and make it so that a persons name is different in different places.…

> IN MONGODB YOU HAVE TO WRITE YOUR CONSTRAINTS IN THE CODE AND THAT IS NOT THE CASE WITH SQL.

You moron, that's not a good thing.

Post reply on HN