MongoDB queries don’t always return all matching documents • Hacker News Board

Re: MongoDB queries don’t always return all matching documents

#271

Post by Kiro » Wed, Jun 08, 2016, 9:27 AM UTC

eloff wrote: post #41

im_down_w_otp wrote: post #16

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

> Look I get that's it's easy to use and easy to get started with

Yes, and that's why I'm using it.

> but you're going to pay for all of that later. [...] When things get busier you're in for a world of pain.

Will never happen.

Re: MongoDB queries don’t always return all matching documents

#272

Post by erez » Wed, Jun 08, 2016, 9:28 AM UTC

derefr wrote: post #237

arctor wrote: post #233

Earlier quoted context omitted.

The problem is, in a great portion of real world projects "eventually" never comes and there's just no time for any major refactoring or replacing technologies since you are too busy implementing the feature that was needed two weeks ago.

I've often dreamed of a specific type of software built and released as "prototypeware", where any app created using it will have certain built-in scaling limits—and going past them will irrevocably force the app into a read-only mode. It would warn anyone monitoring it well in advance of hitting such a limit, of course. But there'd be no way to just slide the limit upward or otherwise tarry. It'd force the migration…

This will not work. The whole "prototype" idea assumes once you grow out of the "prototype" phase you have the time, money, manpower, etc. to rewrite the whole thing based on solid, powerful technology and tools. That is, more of then than not, not the case.

The first problem is that every tool has demands, especially the limited ones, and you end up writing your application around those limits and demands, using platform-specific code that will have to be discarded and re-written come the migration.

The second problem is that these tools dictate design, and once you try migrating, you still have an application designed around the prototype tools, which make a lot of concessions and have design flaws because of that.

Finally, I've never understood the need for learning a specific tool, platform or language for "rapid prototyping". Use the tools you will use eventually, it's not that building something in, say, Java from scratch will take an order of magnitude more time and effort than building it on Node.js, despite all the hype, especially if you're a Java shop.

Re: MongoDB queries don’t always return all matching documents

#273

Post by fortytw2 » Wed, Jun 08, 2016, 9:34 AM UTC

dijit wrote: post #239

manigandham wrote: post #175

Earlier quoted context omitted.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps. Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Well, that's not entirely true. if you're on a mac you can download postgresql.app[0] which produces a small icon in the top right status bar. You don't have to install users or permissions or anything it's super easy to set up. Getting it on prod can come later but for the first five minutes it works. (granted, this neglects contrib extensions like hstore) [0] http://postgresapp.com/

That sets up postgres. It doesn't let you get started doing CRUD operations inside your postgres though - which is where MongoDB shines, "just store my data, fuck it"

Re: MongoDB queries don’t always return all matching documents

#274

Post by 5350-uiop-1130 » Wed, Jun 08, 2016, 9:37 AM UTC

nevi-me wrote: post #218

5350-uiop-1130 wrote: post #137

Is MongoDB really that bad? I am someone just getting into Meteor Js and it seems like moving from MongoDB would make it Meteor trickier to learn. Is it difficult to switch to an alternative? Thanks

It's not, go ahead and use it, learn and gain experience. It's not a replacement for SQL databases. It doesn't have joins and the biggest issue academics and as sysadmins have is it's not fully ACID compliant, so no transactions for example. If I was writing this 2 years ago, I would say horizontal scaling is much easier. Add a node to your replica, watch it catch up, and continue. Have data stored in an array 4 leve…

Thanks

Re: MongoDB queries don’t always return all matching documents

#275

Post by chx » Wed, Jun 08, 2016, 9:48 AM UTC

jeffdavis wrote: post #187

chx wrote: post #174

Earlier quoted context omitted.

This will be highly subjective but you need to get over the "postgresql has the longest feature list so why don't you use it". The last startup I have been involved with tried to use PostgreSQL and needed to move to MySQL (yeah, well) because commercial support was both more expensive and less useful than what we were able to get for MySQL. Perhaps today it's different. While I no longer use PostgreSQL much, every ti…

"just last month I found MySQL, heck even SQLite supports triggers with code inlined into the trigger body but PostgreSQL mandates writing a separate function for the trigger" The right response, as a postgres developer, is to agree that you describe a useful feature, and perhaps implement it to help other users. But my advice to you is to be willing to put up with some short-term annoyances. Sometimes the best choic…

It is listed on the TODO https://wiki.postgresql.org/wiki/Todo page, apparently since 2012. I haven't coded in C since 1998, I do not think you want me to touch the PostgreSQL code base.

Re: MongoDB queries don’t always return all matching documents

#276

Post by blackoil » Wed, Jun 08, 2016, 9:49 AM UTC

manigandham wrote: post #175

Pxtl wrote: post #101

Earlier quoted context omitted.

Because postgres doesn't focus on the first-five-minutes experience the way Mongo does. Even the name is hard to say. Mongo is a dumb, dead-end platform, but they know how important ease-of-use is.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps. Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Postgres members should listen this and have a simple getting started guide for osx, Windows, Linux. I tried brew install postgresql. There was no single place which tells me how to start server, access command line, create db etc.

Re: MongoDB queries don’t always return all matching documents

#277

Post by nrinaudo » Wed, Jun 08, 2016, 10:08 AM UTC

Lazare wrote: post #124

spriggan3 wrote: post #76

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

I often liken NoSQL databases to dynamically typed languages.

With a NoSQL database, you have an implicit schema, but it will only be enforced and fail at runtime - when your code expects a field but failed to find it, for instance.

With a dynamically typed language, you have implicit types, but only enforced at runtime - when your code expects a value to be an int but finds a boolean, for instance.

And both are fine, there is a need for both. I can see how the flexibility of being able to change, well, everything by just flipping a switch in your head ("this is an int now") might be helpful for, say, data exploration problems.

It's just that in a production environment, these features of NoSQL databases and dynamically typed languages turn into massive sources of problems and oh god, just don't.

Re: MongoDB queries don’t always return all matching documents

#278

Post by pluma » Wed, Jun 08, 2016, 10:08 AM UTC

rco8786 wrote: post #13

lossolo wrote: post #4

I've just migrated one project from mongo to postgresql and i advise you to do the same. It was my mistake to use mongo, after I've found memory leak in cursors first day I've used the db which I've reported and they fixed it. It was 2015.. If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db…

> If you have a lot of relations in your data don't use mongo Perhaps you should have been using a relational database from the get go. Sounds more like your issue, not mongo's.

Contrary to what the name implies, most relational databases don't handle querying relation-heavy data well. If you need to hit plenty of relations in your queries, instead consider something that is optimized for that, like a graph database (or multi-model including graph).

Re: MongoDB queries don’t always return all matching documents

#279

Post by joncrocks » Wed, Jun 08, 2016, 10:22 AM UTC

gaius wrote: post #257

catnaroek wrote: post #232

Earlier quoted context omitted.

> I have absolutely no idea what a "SQL migration" Transforming data to conform to a new schema. It's a tedious process that could use some automation, but ditching schemas because data migrations are tedious is like ignoring traffic rules because you're in a hurry. > I assume it's just a bogeyman made up by the MongoDB snake-oil salesmen. It's just like dynamic type marketing: “If you never define the structure of y…

Oh, that doesn't happen much, because serious organizations have dozens of apps connected to a DB and no-one would be crazy enough to big-bang change them all at once to use a different schema. Adding columns and tables tho' is easy, populating new tables from existing, replacing tables with views, no problem.

I'd argue the inverse. Serious organisations shouldn't have lots of different apps reading from each other's databases, for the reason you suggest above. It makes changes difficult and the application landscape fragile. It also encourages not having well-documented interdependencies between your apps (and consequently no way of knowing what the impact of any given change to an app will have).

It's not uncommon however in large organisations with mature applications, without strict IT policies.

Complex applications that are undergoing active changes shouldn't be constrained in this manner, and in that context a SQL migration is more of an in-place upgrade of the data structures that hold the application data between different versions of the application.

Adding columns, adding tables, replacing columns with other columns and managing how the data should look in the new schema, based on the 'old' data structures.

Re: MongoDB queries don’t always return all matching documents

#280

Post by DoofusOfDeath » Wed, Jun 08, 2016, 10:28 AM UTC

hfaran wrote: post #221

1879325235 wrote: post #204

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> None of you have any idea what you are talking about even though you are supposed to be hackers. You heard it here first folks. None of us know anything.

> You heard it here first folks. None of us know anything.

You've just taken the first step toward's web-scale enlightenment.