Live data from Hacker News

Goodbye MongoDB, Hello PostgreSQL (2015)

developer.olery.com

71–80 of 172 posts

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#71
post #12

Earlier quoted context omitted.

These days MongoDB is mostly a subset of what SQL servers do. If all you need is a document store you can simply make a table with a JSONB column and get that. If you later discover you do need relations you can do that with e.g. Postgres but not with MongoDB. Historically MongoDB has been known for dropping or mangling data in some unfortunate ways, I'm not entirely sure what the story there is these days but person…

A single point for Mongo over Postgre JsonB is WiredTiger is very good at compression. In times of Mongo:4.0 vs Postgre:10.5 it was like 20Gb vs 50Gb. Though could depend on documents structure. Not sure about current state. Querries were also somewhat faster in Mongo.

> A single point for Mongo over Postgre JsonB is WiredTiger is very good at compression

Granted, the compression story in postgres isn't currently amazing.

> Querries were also somewhat faster in Mongo.

If you really want fast queries it's not super hard to tune postgres to be fast and drop data if it crashes. It's a bit trickier to tune it to be quite that fast if you want to keep all your data, but for a single table of json documents it should be doable if you're okay with not having transactions.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#72

There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…

> During development you constantly add columns and it's a pain to make sure that this column is added to the sql databases on development, testing, staging, prod etc. With mongo, you just add a column and you are done.

I have found liquibase (https://www.liquibase.org/) to be extremely helpful for this.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#73
post #29

I use MongoDB as a front-end that does full-stack prototypes from time to time in Node, and it's great using javascript from start to finish, you don't have to switch mental gears, so creating an MVP it's fast... but. Sometimes I feel like I'm just avoiding SQL for no reason, not using any of the advantages of NoSQL, and only using Mongo as a SQL-wannabe with a shoehorn like mongoose. I wonder how many MongoDB projec…

The idea of "just dumping JavaScript objects in there" is one of the very pitfalls of the technology.

I also regularly used NoSQL (Mongo, Firebase) on the premise that I was prototyping and it was faster. I thus built an enterprise app which has grown within a large company, and there's barely a day I don't feel insecure about the NoSQL db powering it, because the underlying data model is actually relational.

What field did I use as a "foreign key"? Are these "foreign keys" which are replicated in several places in sync? Will a change in one of these fields break the entire application? What is even the schema, and how do I "migrate" it as requirements change?

These are all questions which I now ask myself regularly when maintaining and extending the app, and which clearly hinder my development speed. The data model is not even that complex - around four main entities - but on the other hand the application code has grown significantly, and the effects of technical debt are thus compounding.

So, in a nutshell, any speed I gained at the beginning was undone later on.

And in line with your reflection, we often conflate the dev speed of NoSQL with an inherent property of the technology, when in fact what's happening is that we're simply more familiar with it. I'm all for "build in what you know", but if you want to build anything semi-serious, you should probably know SQL.

Granted that SQL forces you to think more deeply about your abstract data model ahead of time and how that materializes into a concrete schema. But I don't think one should ever shy away from this exercise, as it is a fixed time investment that will pay huge dividends down the line.

In other words: SQL is slower in that you have to devote some time planning and designing your data model at the beginning of the project. This may be five, twenty or a hundred hours depending on the complexity of what you're building. But this is largely a fixed investment which will dilute and make you faster over time, as operations on data thereafter actually become much easier and not harder. Other reasons why this is a worthwhile investment is because it can be quite fun, and because I believe it generally makes you a better application developer: data design should precede the implementation of logic, no matter what you're building.

> Don't get me wrong, I love MongoDB for a lot of weird stuff (like web scraping, CSV files, and archive data

PostgreSQL (and possibly other RDB technologies) allow you to dump JSON objects in the DB too - and in the case of Postgres this a mature API as far as I know. So you can opt for a mixed data model where the clearcut parts (eg, users and ownership) are relational, while others (eg, results of scraping, or other ever-shifting unstructured data) can be schemaless.

In summary: if your data is relational, go for a relational database technology, even for prototypes.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#74

There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…

> Happy to answer followup questions. This thread is like a piñata that gets pulled out every 6 months or so take a wack at MongoDB and talk about how everyone should just use PostgreSQL for every database problem no matter whether it's relational or not. No-one cares about whether MongoDB is useful or not.

"This thread is like a piñata"

Cudos for having master class control of the word. My favorite punch bag are play stores, root of all evil.

A list of cycling subjects that split opinions about 50-50 in this news group would be hilarious.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#75
post #7

MongoDB and Postgres have very different strengths and weaknesses. It sounds like they didn't consider the needs of their read and write patterns before they decided to go with MongoDB and then blamed it for not being a good fit for their use case. The article doesn't give any insight at all into the cause of the performance issues, just that they happened. It could've been that they were doing something inefficientl…

The article doesn't give any insight into the cause of the performance issue because they coudn't work out what that cause was: In another instance we noticed degraded performance of our applications and managed to trace it to our MongoDB cluster. However, upon further inspection we were unable to find the actual cause of the problem. No matter what metrics we installed, tools we used or commands we ran we couldn’t f…

True, but it seems like they blamed it on MongoDB just being bad without justification and proposed Postgres as the solution.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#76

There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…

> If I am building a financial product which directly handles money or money instruments (bank, stock trading) I would definitely not use mongo for that.

My friend used to work for citi bank, his team managed high value transactions (in billions). I was _stunned_ to hear that they were storing all their data in MongoDB.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#77

There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…

Why do you consider sharding to be marketing fluff? If your dataset was big enough wouldn't that be a requirement for your database?

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#78
post #57
post #17

Earlier quoted context omitted.

In what scenarios does one work on data without any relations? Honest question - perhaps I'm too used to think in SQL but even with e.g. a completely flat bunch of documents one usually needs at least access control (i.e. document owners, which is a relation).

Financial data In basically every case you are querying all data for a symbol in a given timeframe and then process it further outside of the database.

What financial systems are you designing?

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#79

There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…

> If I am building a financial product which directly handles money or money instruments (bank, stock trading) I would definitely not use mongo for that. My friend used to work for citi bank, his team managed high value transactions (in billions). I was _stunned_ to hear that they were storing all their data in MongoDB.

Citibank is a leading expert in losing your money, MongoDB is a perfect fit for their business model.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#80
post #57
post #17

Earlier quoted context omitted.

In what scenarios does one work on data without any relations? Honest question - perhaps I'm too used to think in SQL but even with e.g. a completely flat bunch of documents one usually needs at least access control (i.e. document owners, which is a relation).

Financial data In basically every case you are querying all data for a symbol in a given timeframe and then process it further outside of the database.

Mongo is obviously not competitive with KDB though, if you are talking of timeseries data as you suggest.

I would think it is in fact very much the wrong tool for the job. Time series data screams structured storage, not unstructured.

Post reply on HN