Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL is enough (2024)

gist.github.com

Post reply on HN
Print view View original

71–80 of 97 posts

Re: PostgreSQL is enough (2024)

#71

Post by jamesfinlayson »

ronbenton wrote: post #44

I have worked some places where a significant amount of business logic lived in database functions and triggers and, hoo boy, that was really hard to reason about. If you're disciplined enough to have migrations around that implemented all that stuff, you're still going to have an unpleasant time piecing all of it together when debugging. But often you're in a state where you don't even have those breadcrumbs and you…

Me too, and it put me off ever using stored procedures or triggers. I've worked at places where stored procedures are big but they were much more organised about it.

Re: PostgreSQL is enough (2024)

#72

Post by jamesfinlayson »

whh wrote: post #9
throwaway7783 wrote: post #8

Earlier quoted context omitted.

Re: bad design decisions - This can be said about any technology.

+1 - when to prematurely optimise, when not to.

Oh yeah - I remember working with a guy who absolutely insisted that we build application-level sharding into what we were working on (i.e. each primary key would have some metadata that our application logic would use to route traffic to the correct Postgres instance). Our database was not big - 10s of gigabytes - but Instagram did something similar and we needed to be able to scale. I pushed back hard, and fast forward two years and the application was decommissioned without hitting any sort of Postgres bottleneck.

Re: PostgreSQL is enough (2024)

#73

Post by rmunn »

frollogaston wrote: post #46
danpalmer wrote: post #21

Lots of the alternatives that this site claims Postgres will do are things you'd only consider well past the point that Postgres would be viable. Kafka? No one wants to operate Kafka, if it's a serious contender it's because you need things only it can do. Same with Elasticsearch, it sucks to operate, sucks to build a second stack just for search, so you'd only consider it at the point that Postgres is no longer suit…

Never used Kafka, mostly cause I've been afraid to learn something called Kafka.

You don't want to know what kind of bugs (massive bugs) you'd run into.

Re: PostgreSQL is enough (2024)

#74

Post by cortesoft »

The page makes an argument that having a bunch of disparate databases doing specialized things means you have a higher maintenance burden, since you are maintaining more things and will be paged for more failures, but in my 20+ years maintaining production infrastructure, I find that it is often much more difficult to maintain one large database than multiple smaller ones.

You have pushed your entire infrastructure into a single failure domain, for one thing. You make it certain that if your database fails, EVERYTHING fails.

In addition, there is resource contention and workload variability. As you start to push your postgres instance, all the workloads hitting your database are going to be competing for resources. While postgres itself is good at parallelizing the work it is doing, all that work is still going to be hitting the same database, and competing for the same kcache. Your entire infrastructure might degrade in performance at the same time.

Any issue with one component can cascade very easily if they all share the same database. If your login functionality has a bug and is creating churn on your database, it can lock everything.

With multiple databases, you have a much smaller blast radius when you do database operations. You isolate your workloads and can independently scale them.

Admittedly, all of these issues occurred at a place that had high traffic and high availability requirements. Honestly, though, if your load is so low that you never feel infrastructure pressure, it probably doesn't matter what strategy you use.

Re: PostgreSQL is enough (2024)

#75

Post by threatofrain »

SwellJoe wrote: post #13

I'd go further, and say that most of the time, "SQLite is enough". But, yes, PostgreSQL is all I ever use for anything that needs to be big. I ported a big old web app that had ScyllaDB, Elastic Search, Redis, and probably some other stuff I've forgotten. It got PostgreSQL+PostGIS (it's a mapping app), that's it. I'm sure there's some situation where it would be worth looking at all that other stuff, but it's ridicul…

For database I think principle of medium fit is better than principle of least power. Flexibility for common circumstances is more important than making sure you made the most petite choice possible.

Re: PostgreSQL is enough (2024)

#76

Post by nemothekid »

eximius wrote: post #26
danpalmer wrote: post #21

Lots of the alternatives that this site claims Postgres will do are things you'd only consider well past the point that Postgres would be viable. Kafka? No one wants to operate Kafka, if it's a serious contender it's because you need things only it can do. Same with Elasticsearch, it sucks to operate, sucks to build a second stack just for search, so you'd only consider it at the point that Postgres is no longer suit…

Is that not kinda the point? People reach for the things you mentioned wayyyyyy before they should. Just because you want something queue shaped or search, doesn't mean you should reach for the big, specialized, expensive technology, when Postgres can already support it in your existing infrastructure up until some significant scale you often won't surpass.

>People reach for the things you mentioned wayyyyyy before they should.

Most orgs are probably due for a technical reassessment. Scale that I believe would have melted postgres in the past now just runs fine. CPUs got a ton faster and memory got cheaper.

Projects that would have required a complex distributed setup 10 years ago can now be handled by a single machine. For a lot of use cases, CPUs got faster than population growth.

Re: PostgreSQL is enough (2024)

#77

Post by ammo1662 »

SwellJoe wrote: post #13

I'd go further, and say that most of the time, "SQLite is enough". But, yes, PostgreSQL is all I ever use for anything that needs to be big. I ported a big old web app that had ScyllaDB, Elastic Search, Redis, and probably some other stuff I've forgotten. It got PostgreSQL+PostGIS (it's a mapping app), that's it. I'm sure there's some situation where it would be worth looking at all that other stuff, but it's ridicul…

Yes, for most projects, my path is SQLite -> PostgreSQL. For monolithic applications, SQLite is usually good enough. I've used OLTP + OLAP database setups before(SQL+ELK), but both data synchronization and operational overhead were very high. Before adopting such a solution, you really need to ask yourself: do you actually have that many users?

Re: PostgreSQL is enough (2024)

#78

Post by keynha »

nine_k wrote: post #48
anitil wrote: post #29

Earlier quoted context omitted.

I feel like any problem that Postgres can't handle is a good problem to have. Either you've got so many customers that you're hitting sharp edges, or you're working on such an interesting problem that you're out of the domain where Postgres is helpful. That I should be so lucky

A moderate-size queue is already a problem that Postgres struggles to handle %) But fortunately such problems are rare, and relatively well-known, like, say, doubly-linked lists in Rust. For most other problems, Postgres works well, or at least well enough.

[flagged]

Re: PostgreSQL is enough (2024)

#79

Post by fyredge »

cortesoft wrote: post #74

The page makes an argument that having a bunch of disparate databases doing specialized things means you have a higher maintenance burden, since you are maintaining more things and will be paged for more failures, but in my 20+ years maintaining production infrastructure, I find that it is often much more difficult to maintain one large database than multiple smaller ones. You have pushed your entire infrastructure i…

Seems to me that a single database does reduce maintenance burden, just at the cost of higher risk of failure. The trade off should be for the architect to decide.

Re: PostgreSQL is enough (2024)

#80

Post by jamesfinlayson »

overflowy wrote: post #23

Moving business logic into database functions is the shortest path to insanity.

I remember an old colleague telling me at a previous job they'd moved all business logic into triggers and stored procedures because their database was on their most powerful server. And then one day it wasn't and the database started to choke horrifically.
Post reply on HN