Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

Post reply on HN
Print view View original

351–360 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#351

Post by tomp »

danbmil99 wrote: post #314

Oh, the fud of it. The behavior is well documented here https://jira.mongodb.org/browse/SERVER-14766 and in the linked issues. Seasoned users of mongodb know to structure their queries to avoid depending on a cursor if the collection may be concurrently updated by another process. The usual pattern is to re-query the db in cases where your cursor may have gone stale. This tends to be habit due to the 10-minute cursor…

> The usual pattern is to re-query the db in cases where your cursor may have gone stale.

You mean, every time you query the DB? Do you also need to re-query the re-queries?

Re: MongoDB queries don’t always return all matching documents

#352

Post by matwood »

Lazare wrote: post #124
spriggan3 wrote: post #76

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

To add to your point about schemas. The new generation has not learned that the data almost always outlives whatever throwaway front end was written to work with said data. Tying the data to some sort of flavor of the month framework is setting up for all sorts of pain later.

I despise mysql, but even it is better than mongo. At least with it I can easily transition the data to many different uses.

Also, and a point I find amusing is that many users of nosql claim schemaless and then go and write a layer on top of the datastore to enforce a schema. It would have been so much simpler to use a RDMS out the gate instead of badly implementing one.

Re: MongoDB queries don’t always return all matching documents

#353

Post by nrinaudo »

1879325235 wrote: post #204

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

I'm not going to comment on the content of your posts - others have done that. But you might want to check your tone. You sound like a petulant teenager offended by the mere notion that someone might think you wrong.

It honestly makes it really hard not to dismiss whatever you say out of hand. If you don't really care about people taking you seriously then by all means keep it up, but if you're trying to participate in a grown up conversation, you might want to stop throwing tantrums. You know, the basic rule of human interactions, show people the respect you expect them to show you?

Re: MongoDB queries don’t always return all matching documents

#354

Post by KMag »

zeemonkee3 wrote: post #199
im_down_w_otp wrote: post #16

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

NodeJS + MongoDB is this generation's Laurel and Hardy stack ("look at this mess you got me into"). Last generation's was PHP + MySQL.

I actually wonder what the correlation is between PHP use and MongoDB use. They both have an attitude that mistakes ease with simplicity, a philosophy that puts correctness way down the priority list, and an easy introduction with a heavy ongoing maintenance tax.

Re: MongoDB queries don’t always return all matching documents

#355

Post by jeltz »

tremon wrote: post #345
jeltz wrote: post #342

Earlier quoted context omitted.

A common example is making sure that every event in a bookkeeping system balances to zero (e.g. three rows: -125 EUR bank, +100 EUR office materials, +25 EUR tax). To enforce this at the database level you either need to run at SERIALIZABLE, take a table lock, or do something horrible with your database structure (like putting all rows of the event in a JSON blob). When I have solved this case I have done it in the a…

A common example is making sure that every event in a bookkeeping system balances to zero (e.g. three rows: -125 EUR bank, +100 EUR office materials, +25 EUR tax). To enforce this at the database level you either need to run at SERIALIZABLE, take a table lock, or do something horrible with your database structure (like putting all rows of the event in a JSON blob). Or you deny direct table inserts, and provide a stor…

Which is exactly what I did when I had to solve the problem. :) And you are right in mentioning that the fact that you can restrict modification of the database to only certain stored procedures is a great help here.

Re: MongoDB queries don’t always return all matching documents

#356

Post by Jweb_Guru »

1879325237 wrote: post #245

This is exactly what I thought. You are a DBA. You are not a programmer. Whenever I read these threads I think "either these people are all DBAs or they have an extreme emotional attachment to a particular database". There it is - this is a thread full of DBAs rebelling against a database that takes away their power and responsibility.

The people who work on databases are (in my estimation) usually up there with those who work on operating systems, compilers, and video games. Most programmers are simply not dealing with very interesting constraints in terms of latency, extensibility, throughput, storage, concurrency, or other challenging requirements, but database people are. Don't be irritated when they ask for better tools.

Re: MongoDB queries don’t always return all matching documents

#357

Post by cachemiss »

zamalek wrote: post #308
cachemiss wrote: post #163

My general feeling is that MongoDb was designed by people who hadn't designed a database before, and marketed to people who didn't know how to use one. Its marketing was pretty silly about all the various things it would do, when it didn't even have a reliable storage engine. Its defaults at launch would consider a write stored when it was buffered for send on the client, which is nuts. There's lots of ways to solve…

I really agree with your sentiments, that first paragraph is a great quote. I grew quite an adverse to MongoDB after researching it. While I never found this specific caveat, I found other very worrying decisions. > reliable storage engine By "reliable" I assume you mean "consistent?" While MongoDB claims that it's CP (which it's not, as per the article) there's nothing wrong with inconsistent databases (AP, e.g. Cou…

I actually mean reliable. Its probably different now, but at launch, the defaults were fsync'ing every 30 seconds or so. It would literally just apply the change to an memory mapped buffer and just fsync it once in a while.

They did that so they could look good in benchmarks, and it's why they recommended so strongly that your memory completely fit in RAM or else things would fall apart (pro-tip, any system that recommends that has a poorly designed storage engine).

They also screwed up the consistent side of things as well.

Re: MongoDB queries don’t always return all matching documents

#358

Post by matwood »

1879325237 wrote: post #250
Lazare wrote: post #223

Earlier quoted context omitted.

> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" f…

> I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" fields into a single "name" field with Mongo or a SQL DB is basically that the SQL DB has better tooling so it'll be cl…

> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

You must really not understand SQL. SQL is one of the easiest languages to work with sets of data.

Are you one of those people who pull back 100k records from the db in order to get a random 25 using a 'real' programming language?

Re: MongoDB queries don’t always return all matching documents

#359

Post by matwood »

hk__2 wrote: post #95
spriggan3 wrote: post #76

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

It’s useful for prototyping. When you don’t know which schema you’ll end up using having an *SQL database is tedious because you have to do migrations every time you change the schema. Once you’re done prototyping you can switch to a better alternative.

Use SQLite and let the tool drop and recreate all the tables during prototyping. No migrations required.

Re: MongoDB queries don’t always return all matching documents

#360

Post by matwood »

manigandham wrote: post #175
Pxtl wrote: post #101

Earlier quoted context omitted.

Because postgres doesn't focus on the first-five-minutes experience the way Mongo does. Even the name is hard to say. Mongo is a dumb, dead-end platform, but they know how important ease-of-use is.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps. Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Mongo also goes to every conference and pitches themselves constantly. Maybe postgres needs a marketing person :)
Post reply on HN