Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye MongoDB, Hello PostgreSQL (2015)

developer.olery.com

Post reply on HN
Print view View original

121–130 of 172 posts

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#121

Post by samlambert »

bushbaba wrote: post #32
srich36 wrote: post #24

Earlier quoted context omitted.

There are plenty of modern, distributed RDBMSes that make sharding transparent to the user (E.g. cockroachDB, yugabyte, vitess, many cloud offerings, etc.). Most NoSQL databases end up adding transactions because they are important, and thus the scale advantages for NoSQL systems over relational databases are diminishing, if remaining at all.

Latency at p99 is substantially better at scale with a key value lookup in nosql over postgresql. Thing is most people don’t have scale these days. You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd. Until you exceed that you don’t necessarily have scale.

Well it depends what you consider scale. When you look at the high scale deployments like Slack, YouTube, GitHub, Square, etc they are all using Vitess and MySQL.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#122

Post by psychometry »

anticristi wrote: post #5

I used both NoSQL and SQL databases. Proper replication topologies (e.g. multi-master configurations) and JSON columns killed the NoSQL movement. I'm glad the NoSQL movement existed and forced the SQL camp into out-of-the-box thinking, but I wouldn't go for any NoSQL solution today on a greenfield project.

No SQL database has the anywhere near the search capabilities of Elasticsearch/Lucene. I also wouldn't choose Postgres for storing logs, either, but I would use a document store (i.e. ELK stack).

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#123

Post by philliphaydon »

smt88 wrote: post #27
StreamBright wrote: post #20

These articles never get old. Hey we picked solution A and realized that it has some cons not only pros and now we are moving to solution B that only has pros. Few years later there is a new article, hey we are moving to solution A|C because...

I've never seen an article about switching from Postgres to something else. Obviously Citus, Timescale, and other Postgres forks don't count. Uber did one a while ago, but tl;dr they didn't really understand Postgres very well and were dealing with unique problems.

I'm not sure they understood the problem they were trying to solve let alone understood Postgres.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#124

Post by bushbaba »

samlambert wrote: post #121
bushbaba wrote: post #32

Earlier quoted context omitted.

Latency at p99 is substantially better at scale with a key value lookup in nosql over postgresql. Thing is most people don’t have scale these days. You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd. Until you exceed that you don’t necessarily have scale.

Well it depends what you consider scale. When you look at the high scale deployments like Slack, YouTube, GitHub, Square, etc they are all using Vitess and MySQL.

YouTube doesn’t use just mysql or vitess these days they migrated to other google databases.

Slack, GitHub both are stupidly shardable. I doubt it’s one RDBMS handling every customer, chat room, git repo. And instead they’ve segmented the workload across multiple instances.

That doesn’t work for every use case

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#125

Post by bushbaba »

threeseed wrote: post #68
bushbaba wrote: post #32

Earlier quoted context omitted.

Latency at p99 is substantially better at scale with a key value lookup in nosql over postgresql. Thing is most people don’t have scale these days. You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd. Until you exceed that you don’t necessarily have scale.

> You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd And then that box falls over because the entire region fails like just happened yesterday with OVH. Or it just randomly fails like has happened to me with AWS dozens of times. Vertically scaling a database on your own cloud instances is amateur hour. Either use a cloud-managed database or one that is highly avai…

Sure. And those services support some fairly beefy instance types. Such as https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-rd...

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#126

Post by neogodless »

fhunt wrote: post #96

@dang, I found this in the guidelines: Please don't post shallow dismissals, especially of other people's work. I see good critical comments here but also many shallow dismissals from people who just sneer at the work of all software engineers who created MongoDB. Why is such behavior tolerated in any Mongo thread and why do these people never face any consequences?

> Why is such behavior tolerated in any Mongo thread and why do these people never face any consequences?

I think you answered your own question: "guidelines."

Those that believe in this community, and want adherence to the guidelines will vote on submissions and comments so as to reflect that belief. Shallow dismissals tend to get voted down, and they may be refuted if someone is inspired to put the effort in to explain why such a shallow dismissal is either unwelcome or missing important information. Seeing as these are guidelines, there is no swift and absolute retribution for light infractions, when the community can largely self-correct such things.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#127

Post by JeremyNT »

HelloNurse wrote: post #93
threeseed wrote: post #66

Earlier quoted context omitted.

Most data in large enterprises e.g. telcos, banks, insurance etc are not stored in a single data warehouse with well-defined relationships. They are in multiple disparate silos of which their EDW is just one. It's why Data Lakes become so commonplace because it was an easy way to just get all of the data out of the silos into one place so that the business could attempt to join between them. And it's why MongoDB (and…

If an enterprise wants to "get all the data out of the silos", unstructured storage is only "friendly" to the temptation to be sloppy in such an export. Not only missing or corrupted individual columns, but completely incoherent and arbitrary complex document structures. If "no-one knows the schemas and no-one knows how to join between them" the data lake project has already failed to provide value.

Yeah, it's kicking the can down the road!

If you can't or won't do the work to understand and define the schema at the time of ingestion, you're asking consumers of that data to do the work for you later on, and then you're looking at an archeology project to try and figure out what the heck people were thinking in the past. It's only going to get worse as you keep shoving new data in there with its own incompatible poorly defined schema.

It probably ticks a compliance box but good luck making use of it...

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#128

Post by aidos »

richardw wrote: post #106
aidos wrote: post #35

Earlier quoted context omitted.

I was involved in those original discussions and it was just the standard “how to manipulate sql” discussion. Prepared statements don’t help you here, right? My understanding is that they’re purely for performance as the parser doesn’t need to be rerun each time. You need to recreate them for every dB session.

No sql injection, can set permissions for each proc, isolates app from DB implementation so you don’t have to redeploy app. Eg you can migrate older data to an archive table and query it on certain circumstances. App doesn’t need to know.

That’s stored ptocedures you’re talking about - I think. We’re talking about prepared statements.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#129

Post by HelloNurse »

JeremyNT wrote: post #127
HelloNurse wrote: post #93

Earlier quoted context omitted.

If an enterprise wants to "get all the data out of the silos", unstructured storage is only "friendly" to the temptation to be sloppy in such an export. Not only missing or corrupted individual columns, but completely incoherent and arbitrary complex document structures. If "no-one knows the schemas and no-one knows how to join between them" the data lake project has already failed to provide value.

Yeah, it's kicking the can down the road! If you can't or won't do the work to understand and define the schema at the time of ingestion, you're asking consumers of that data to do the work for you later on, and then you're looking at an archeology project to try and figure out what the heck people were thinking in the past. It's only going to get worse as you keep shoving new data in there with its own incompatible…

I've only ever witnessed consumer-driven ad-hoc integration initiatives, where the team that needs data coerces the team that has data into providing it in a usable form, and people notice stale or incorrect feeds because they are actually in use. Why should a company invest server resources and development effort on "ingesting" data that nobody cares about in a way that doesn't fit any purpose? Attempting to collect "all" data is meaningless.

Re: Goodbye MongoDB, Hello PostgreSQL (2015)

#130

Post by jb3689 »

"I don't know what was wrong" articles like this come up really often with Mongo. My team manages thousands of Mongo nodes and we have issues, but few are directly due to Mongo. Mongo's documentation could be friendlier to the operator/maintainer, however the tools for debugging Mongo tend to be sufficient (metric counters, various status/state dumps, general Linux tooling).

You need to understand databases folks. There is no way around it. "Porting to Postgres" does not replace understanding of database internals and figuring out how to debug bad access patterns and explain why they are bad. Blindly migrating data seems like one of the worst things you could ever do - you're not fixing the problem that you don't understand how databases work. I get it - ain't nobody got time for that - but it will come back and bite you with your new shiny fancy tool too if you aren't careful

Anyone who brings up something like master-master hopefully realizes they are talking about a completely different problem now with completely different constraints. Expecting the same access pattern will perform just as well in master-master is wishful thinking - it requires careful design to avoid coordination problems. The meat of the story with databases is: coordination is expensive, distributed coordination is even more expensive

Post reply on HN