Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Great Migration from MongoDB to PostgreSQL

infisical.com

Post reply on HN
Print view View original

271–280 of 339 posts

Re: The Great Migration from MongoDB to PostgreSQL

#271

Post by CSMastermind »

thih9 wrote: post #42

Loosely relevant, “the marketing behind mongodb” https://news.ycombinator.com/item?id=15124306

Man, that thread feels prophetic. Since 2017 whenever this comment was written:

> 100% of my friends who have used Mongo/similar NoSQL have given up and had a nasty rewrite back to pgSQL...

I've led three migrations at different companies from Mongo to Postgres, spending months of my life and millions of dollars.

There really should be a sociology study of why so many people/companies used Mongo even well after the downsides were clear.

Re: The Great Migration from MongoDB to PostgreSQL

#272

Post by bognition »

bit_flipper wrote: post #255

I've run Postgres at large scale (dozens of machines) at multiple companies. I've also run MongoDB at large scale at multiple companies. I like both generally. I don't really care about data modelling differences - you can build the same applications with approximately the same schema with both if you know what you're doing. I don't understand how folks seemingly ignore Postgres' non-existent out of the box HA and ho…

You'll never see true support for horizontal scalability in Postgres because doing so would require a fundamental shift in what Postgres is and the guarantees is provides. Postgres is available and consistent. It cannot truly be partitionable without impacting availability or consistency.

When an application grows to such a scale that you need a partitionable datastore it's not something you can just turn on. If you've been expecting consistency and availability, there will be parts of your application that will break when those guarantees are changed.

When you hit the point that you need horizontally scalable databases you must update the application. This is one of the reasons that NewSQL databases like CockroachDB and Vitess are so popular. They expose themselves as a SQL database but make you deal with the availability/consistency problems on day 1, so as your application scales you dont need to change anything.

Context: I've built applications and managed databases on 10's of thousands of machines for a public saas company.

Re: The Great Migration from MongoDB to PostgreSQL

#273

Post by amluto »

bit_flipper wrote: post #255

I've run Postgres at large scale (dozens of machines) at multiple companies. I've also run MongoDB at large scale at multiple companies. I like both generally. I don't really care about data modelling differences - you can build the same applications with approximately the same schema with both if you know what you're doing. I don't understand how folks seemingly ignore Postgres' non-existent out of the box HA and ho…

> This is an area where MySQL is so significantly better than Postgres (because so many large Internet companies use MySQL) that it surprises me people aren't more unhappy with the state of things.

I’m not sure precisely what you mean by “HA”, but, in my experience, out-of-the-box support for the most basic replication setup in MySQL is pretty bad. Just to rattle off a few examples:

Adding a replica involves using mysqldump, which is, to put it charitably, not a very good program. And the tools that consume its output are even worse!

There is nothing that shops with MySQL that can help verify that a replica is in sync with its primary.

Want to use GTID (which is the recommended mode and is more or less mandatory for a reasonable HA setup)? Prepare for poor docs. Also prepare for the complete inability of anyone’s managed offering to sync to an existing replica set via mysqldump’s output. RDS will reject the output due to rather fundamental permission issue, and the recommended (documented!) workaround is simply incorrect. It’s not clear that RDS can do it right. At least Azure sort of documents that one can manually real and modify the mysqldump output and then issue a manual API call (involving the directives that you manually removed from the dump) to set the GTID state.

Want point-in-time recovery? While the replication protocol supports it, there is no first-party tooling. Even just archiving the replication logs is barely supported. Postgres makes it a bit awkward, but at least the mechanisms are supported out of the box.

But maybe the new-ish cluster support actually works well one it’s set up as long as you don’t try to add managed RDS-style nodes?

Re: The Great Migration from MongoDB to PostgreSQL

#274

Post by 999900000999 »

swasheck wrote: post #176
999900000999 wrote: post #174

Earlier quoted context omitted.

>As part of that stack, we chose MongoDB + Mongoose ORM because the combination presented least overhead and allowed us to ship quality features quickly. As Sir Tony Hoare states, “premature optimization is the root of all evil,” and there was certainly no need for further optimization at the time. Looks like the tool did a great job while they were getting started. Mongo is very easy to hack something together with,…

yeah ... in the early days of mongo it was pitched as a replacement for relational because a) it had faster writes without those pesky joins and constraints, and b) because it's too hard to model data, so chances are it's just unstructured. the onslaught of "relational is dead" and "joins are annoying" was tremendous

Plus there's a good chance they didn't know exactly what they needed to build when starting out.

Re: The Great Migration from MongoDB to PostgreSQL

#275

Post by evanelias »

amluto wrote: post #273
bit_flipper wrote: post #255

I've run Postgres at large scale (dozens of machines) at multiple companies. I've also run MongoDB at large scale at multiple companies. I like both generally. I don't really care about data modelling differences - you can build the same applications with approximately the same schema with both if you know what you're doing. I don't understand how folks seemingly ignore Postgres' non-existent out of the box HA and ho…

> This is an area where MySQL is so significantly better than Postgres (because so many large Internet companies use MySQL) that it surprises me people aren't more unhappy with the state of things. I’m not sure precisely what you mean by “HA”, but, in my experience, out-of-the-box support for the most basic replication setup in MySQL is pretty bad. Just to rattle off a few examples: Adding a replica involves using my…

> Adding a replica involves using mysqldump

That's one path, but it is not the only way, and never has been.

MySQL 8.0.17 (released nearly 5 years ago!) added support for physical (binary) copy using the CLONE plugin. And MySQL Shell added logical dump/reload capabilities in 8.0.21, nearly 4 years ago.

Third-party solutions for both physical and logical copy have long been available, e.g. xtrabackup and mydumper, respectively.

And there was always the "shut down the server and copy the files" offline approach in a pinch.

Re: The Great Migration from MongoDB to PostgreSQL

#276

Post by teunispeters »

iambateman wrote: post #65

I use MySQL for everything - always have. Can someone hit me with a few reasons why you would use Postgres over MySQL? I don’t have any familial affinity to any database, but I’m not sure what the benefits to Postgres are relative to MySQL.

Stability, reliability, consistency. If you do anything with Unicode you're also much better in PostgreSQL. Faster indexes, smaller base install (), and much more complete SQL language support.

() note: when PHP was taking off, MySQL had a smaller install base. This has long since changed - PostgreSQL hasn't grown much over the years, and MySQL has, at least since the last time I worked on both circa 2015-ish.

Re: The Great Migration from MongoDB to PostgreSQL

#277

Post by zo1 »

jimbokun wrote: post #256
zo1 wrote: post #225

Earlier quoted context omitted.

From the other side, people like me, see your odd dismissive behaviour in the same light. This bridge and hundreds of other things going wrong throughout the world atm are all symptoms of some sort of rotten problem, and I honestly don't know why you guys are so adamant about dismissing it. Almost as if you want to see it all come crashing down. Pure evil, as Trump would call it.

Is the rate of these incidents increasing, decreasing, or about the same?

Obviously you can say I'm biased but I think they're increasing.

Re: The Great Migration from MongoDB to PostgreSQL

#278

Post by SOLAR_FIELDS »

zachmu wrote: post #246

PostgreSQL really is eating the database world. Although in this case, the authors originally chose an architecture that was poorly suited to their data model. They had relational data and put it in a non-relational store. This was obviously always going to cause problems.

Over the years I’ve learned two things when it comes to picking a database when starting a new project

1 - most data is inherently relational

2 - Postgres is pretty good at basically any problem you throw at it, pretty scalably. It will fall over at bonkers amount of scale in some cases but you probably won’t have bonkers amount of scale

Thus, if you want to pick any new data store that isn’t Postgres for your project, the default is Postgres. You have to specifically convince me that Postgres is not good enough for this specific use case if you want me on board for anything else as a data store.

Re: The Great Migration from MongoDB to PostgreSQL

#279

Post by madsbuch »

jimbokun wrote: post #250
madsbuch wrote: post #213

Earlier quoted context omitted.

Why even use a database? Just assign a variable. All state is dead the second you destroy the container anyways.

Well then single container plus one attached volume.

Yep, my point was that this roughly has the complexity of setting up eg. Supabase.

In conclusion: I would always advice for Postgres over the approach of setting up a volume with sqlite.

The second you need to move on, you need to get that pesky file out of that volume. With Postgres, you just connect to the database from any psql compliant client.

Re: The Great Migration from MongoDB to PostgreSQL

#280

Post by BiteCode_dev »

bit_flipper wrote: post #255

I've run Postgres at large scale (dozens of machines) at multiple companies. I've also run MongoDB at large scale at multiple companies. I like both generally. I don't really care about data modelling differences - you can build the same applications with approximately the same schema with both if you know what you're doing. I don't understand how folks seemingly ignore Postgres' non-existent out of the box HA and ho…

Because vertical scaling can take you so far these days that 99% of companies will never, ever reach the scale where they need more. There is just few incentives.

Especially since:

- Servers will keep getting better and cheaper with time.

- Data is not only in postgres, you probably have redis, clickhouse and others, so the charge is balanced. In fact you may have different dedicated postgres, like one for GIS tasks.

- Those hacky extensions are damn amazing. No product in the world is that versatile.

- Posgres has much better support from legacy frameworks like django/ror/laravel than nosql alternatives. People shits on ORM, but they enable a huge plugin well integrated ecosystem that makes you super productive, and PG is happily and transparently handling all that.

- If by some miracle you actually reach the point you need this, you'll have plenty of money to pay for commercial HA/sharding, or migrate. So why think about it now?

Post reply on HN