Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

Post reply on HN
Print view View original

331–340 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#331

Post by tremon »

Lazare wrote: post #223
1879325235 wrote: post #204

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" f…

what relational DBs are very good at is enforcing foreign key constraints. If your data is relational at all [..]

Nitpick: the "relational" in "relational database" is not a reference to the concept of foreign keys, but to "relational algebra", the mathematical model for regular data structures that SQL is based on.

https://en.wikipedia.org/wiki/Relational_algebra

Re: MongoDB queries don’t always return all matching documents

#332

Post by collyw »

varunom wrote: post #300
collyw wrote: post #246

Earlier quoted context omitted.

For me that is the most important aspect of starting / designing an application. If the data model is accurate, then the code falls into place easily. If its not quite right, more and more code ends up in the application trying to make up for the poor data model.

I second this. My first task in any project is to design the whole data model based on current requirements and while designing it I think of the interfaces and how would they read and write data (to refine requirements). Writing views and actions/APIs on top of well-formed data model then becomes a breeze.

Its so un-agile (but it works).

Re: MongoDB queries don’t always return all matching documents

#333

Post by snaky »

tremon wrote: post #325
jeltz wrote: post #311

Earlier quoted context omitted.

Arbitrary constraints which look at multiple rows require serializable isolation to work under concurrency, and most people do not run their databases at serializable. But you get pretty far with single row check constraints and exclusion constraints (exclusion constraints can be used for checking against range overlap).

Can you give a use-case for such a constraint that can not be avoided by a better data organization? I'm not sure why you would deliberately design your data with constraints based on other row contents. Also: how do you enforce such constraints with MongoDB exactly? If the answer is "do it in the application", then your answer applies to relational databases too.

You don't have to avoid such constraints if they are natural to data model actually. It's much faster (and safer, thanks to transactions) to check new data against some old data when the checking code is as close to the data as possible, instead of make a couple additional requests between DBMS and the business logic layer.

Re: MongoDB queries don’t always return all matching documents

#334

Post by collyw »

PeterisP wrote: post #326
collyw wrote: post #254

Earlier quoted context omitted.

If thats really a problem host your database on a cloud platform and let those guys do the job of a DBA. Works for me at present. I am aware its not going to be a solution for everyone, though it still sounds a lot better than being your own Mongo DBA.

Cloud service doesn't really replace DBA. Yes, it will help you to cover cases like where the server phyically explodes, but that's basically irrelevant, most problems where you need a DBA are caused either by data corruption caused by application code or developer, or performance issues caused by DB structure - in those cases the cloud platform won't do anything for you, they just host the server. They can restore b…

"most problems where you need a DBA are caused either by data corruption caused by application code or developer, or performance issues caused by DB structure"

Is running Mongo going to solve any of those problems? Without a rigidly enforced schema I would guess those problems are going to be amplified rather than solved.

Re: MongoDB queries don’t always return all matching documents

#335

Post by daveguy »

catnaroek wrote: post #307
1879325237 wrote: post #255

Earlier quoted context omitted.

Hacking is trying to get the SQL representation into something programmers use via the way of an ORM. At the end of the day, a SQL database doesn't represent the data in a way the programmer uses the data. It has to be transformed and that is a bigger hack than loosening some constraints.

ORMs are a really bad attempt to force a square peg into a round hole. The mismatch between the relational model and object-oriented design principles is simply too big. In the relational model: (0) A relation is a collection of tuples of primitive values. Every relation has a relation schema, which determines the arity of its tuples and the type of each tuple component. In other words, the relational model is first-…

I was going to say I wish there was a way to bookmark single comments. Then I realized the reply page is essentially a link to a single comment:

https://news.ycombinator.com/reply?id=11861520&goto=item%3Fi...

If anyone wants to bookmark this one.

Re: MongoDB queries don’t always return all matching documents

#336

Post by collyw »

StavrosK wrote: post #156
catnaroek wrote: post #127

Earlier quoted context omitted.

> Mongo is a dumb, dead-end platform, but they know how important ease-of-use is. By “ease of use”, do you mean “ease of making something that seems to work” or “ease of making something that actually works”? I've never used a schema-free database, and ended up thinking to myself “I'm completely sure this database can't possibly contain garbage data”. Or do programmers simply not care about data integrity anymore?

The single biggest source of grief in our production database has been the one JSON field we used once to avoid adding another table. That goddamn thing has crashed the server so many times with invalid data, that I'm never using anything schemaless again. We recently migrated to a proper table and I'm thanking my lucky stars I finally got rid of that devil.

JSON field? You are lucky. We have a Pickled python object stored in our database. Wonderful for debugging when its pretty much unreadable.

Re: MongoDB queries don’t always return all matching documents

#337

Post by m_mueller »

s_kilk wrote: post #253
m_mueller wrote: post #118

Earlier quoted context omitted.

So my question is then: Why not use CouchDB instead? I don't see what Mongo gives you over that and CouchDB is at least dependable and predictable in its operation.

Is CouchDB still alive? I spent a weekend playing with it in January, but it seemed to be a very quiet project, with the last stable release being almost two years ago.

Most of the activity happens at Couchbase now, the company that the inventor D. Katz founded based on CouchDB technology. You can still use Couchbase for free, but it's possible to pay for support. The coolest thing they have is Couchbase Lite, the mobile version of CouchDB, lets you replicate with your server. I find it a very interesting alternative to Core Data, parse and co. and we use it in production.

Re: MongoDB queries don’t always return all matching documents

#338

Post by pritambaral »

chx wrote: post #174
eloff wrote: post #41

Earlier quoted context omitted.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

This will be highly subjective but you need to get over the "postgresql has the longest feature list so why don't you use it". The last startup I have been involved with tried to use PostgreSQL and needed to move to MySQL (yeah, well) because commercial support was both more expensive and less useful than what we were able to get for MySQL. Perhaps today it's different. While I no longer use PostgreSQL much, every ti…

> just last month I found MySQL, heck even SQLite supports triggers with code inlined into the trigger body but PostgreSQL mandates writing a separate function for the trigger

I found something similar (and in the last month too) – insofar as we're talking missing popular features – but with MySQL's and Postgres's positions reversed.

`ALTER TABLE ... ADD CONSTRAINT CHECK ...` runs on MySQL without an issue, and so does any INSERT or UPDATE violating that CHECK constraint. A bug was filed in 2004.

Re: MongoDB queries don’t always return all matching documents

#339

Post by [deleted] »

gaius wrote: post #215
1879325235 wrote: post #204

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

I have been a DBA for 20 years, Oracle, SQL Server, Sybase, Informix, Postgres, MySQL/MariaDB... About the only big name I haven't had serious experience with is DB2. We did hot-releases into Prod routinely. I have absolutely no idea what a "SQL migration" is or why it would be so hard. I assume it's just a bogeyman made up by the MongoDB snake-oil salesmen. It is certainly not something "traditional" database people…

[deleted]

Re: MongoDB queries don’t always return all matching documents

#340

Post by daveguy »

1879325237 wrote: post #245

This is exactly what I thought. You are a DBA. You are not a programmer. Whenever I read these threads I think "either these people are all DBAs or they have an extreme emotional attachment to a particular database". There it is - this is a thread full of DBAs rebelling against a database that takes away their power and responsibility.

Programmers should have more than a passing understanding of database administration. They should understand regularization / constraints / ACID / SQL / etc. This was core curriculum in my CS undergraduate. More importantly they should understand why they are important and when they are needed. Most of the commenters here are probably programmers who understand these things -- or programmers who have gotten burned by database stew and learned the hard way. It seems like you would to well to take a deep breath, drop the antagonistic view of traditional databases (just another tool), and educate yourself on their use and implementation.
Post reply on HN