Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL is enough (2024)

gist.github.com

Post reply on HN
Print view View original

51–60 of 97 posts

Re: PostgreSQL is enough (2024)

#51

Post by jabwd »

eximius wrote: post #26
danpalmer wrote: post #21

Lots of the alternatives that this site claims Postgres will do are things you'd only consider well past the point that Postgres would be viable. Kafka? No one wants to operate Kafka, if it's a serious contender it's because you need things only it can do. Same with Elasticsearch, it sucks to operate, sucks to build a second stack just for search, so you'd only consider it at the point that Postgres is no longer suit…

Is that not kinda the point? People reach for the things you mentioned wayyyyyy before they should. Just because you want something queue shaped or search, doesn't mean you should reach for the big, specialized, expensive technology, when Postgres can already support it in your existing infrastructure up until some significant scale you often won't surpass.

The amount of redis-as-database, or X really, that I see deployed that practically make no sense is insane. I haven't run the numbers to make my following claim based on anything but my feelings, but just incompetent dev alone is probably what keeps cloud computing alive.

Re: PostgreSQL is enough (2024)

#52

Post by guhidalg »

fastball wrote: post #5

> But the bar should be high: only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative Why do I need to push Postgres to its limits before using a different solution? Throwing a hosted Redis in front of some hot-path API calls is very straightforward and easier to reason about than materialized views or UNLOGGED tables.

Because it's less moving pieces to only have one bit of state to think about.

You already have a connection string to your database with a password or authn/z with your cloud provider. If this is a "serious" application, you have backups, monitoring, user roles, pgbouncer, partitioning, and other Postgres-specific things to think about. With just a little bit of care, you can make whatever queries you are running fast enough to not need redis.

But ok, you think adding redis is going to solve your performance problem because you can just cache API responses in redis instead of hitting the DB. Maybe, but now you have to think about cache invalidation, eviction behavior, sizing the redis instance, another set of authn/z roles to think about, and of course more cost.

I realize we're speaking past each other, but IME Postgres will work well into the terabyte range and if you can't tune your database setup for performance then reaching for cache is a form of premature optimization.

Re: PostgreSQL is enough (2024)

#53

Post by busymom0 »

green_wheel wrote: post #49
busymom0 wrote: post #20

Earlier quoted context omitted.

I recently built a site that aggregates top posts from various sources and am using both SQLite and Swift for my backend. Was a pleasant experience mostly. https://limereader.com/

Why'd you go with swift for the backend?

I am primarily a mobile developer (I am the developer of Hack, hacker news app on iOS and android). While I have developed backends in rust and nodejs, I wanted to give swift a shot to see how it was. I used Vapor for the web server in swift. It was surprisingly pretty good experience and I am considering using it for my next project too.

Re: PostgreSQL is enough (2024)

#55

Post by shepherdjerred »

bitlad wrote: post #6

PostgreSQL is good enough until it's not good enough, when you realize all the bad design decisions that were made before it hits scale. It is the decisions people make around not partitioning, HA, replication that makes it not good enough.

Everything is "good enough until it's not good enough". That's engineering.

Re: PostgreSQL is enough (2024)

#56

Post by AdieuToLogic »

nine_k wrote: post #42
AdieuToLogic wrote: post #31

The first two linked resources: Simplify: move code into database functions Just Use Postgres for Everything Are disqualifying enough to not warrant further reading. A relational database is one form of persistent storage. They are great for managing persistent representations of key abstractions and their relationships. They are not application frameworks nor scalable messaging systems by design.

Did you see that implementation of Quake in CSS? Or Tetris in awk? You can do a similar thing with Postgres, and it will be much less insane. Should you run your entire app stack in Postgres? Likely not. Can you? Likely you can.

> Did you see that implementation of Quake in CSS? Or Tetris in awk?

Do you want to support either of those in production? Get support calls and/or enhancement requests for same?

> Should you run your entire app stack in Postgres? Likely not. Can you? Likely you can.

I have experience with non-trivial systems which have put business logic in database stored procedures. They were not pretty. One example was a single stored proc having north of 40 parameters and was central to a high-volume transaction workflow.

That is how I learned at the time Oracle had a max of 4 threads available to stored procs and that SGA usage can reach out and bite you in "fun and interesting" ways.

Re: PostgreSQL is enough (2024)

#57

Post by danpalmer »

sgarland wrote: post #39
danpalmer wrote: post #21

Lots of the alternatives that this site claims Postgres will do are things you'd only consider well past the point that Postgres would be viable. Kafka? No one wants to operate Kafka, if it's a serious contender it's because you need things only it can do. Same with Elasticsearch, it sucks to operate, sucks to build a second stack just for search, so you'd only consider it at the point that Postgres is no longer suit…

As the sibling comment from eximius mentions, devs will routinely reach for these long before they’re remotely needed. A well-tuned Postgres installation on fast hardware and intelligent schema design can scale incredibly far, even if you’re asking it to double as a message bus and full-text search tool.

Maybe I'm lucky, but I've not seen this. Sure I've seen this from junior developers who aren't thinking through the actual requirements they have, but I've not seen it on a team with any real level of experience.

I have seen many cases of people decrying the use of some of these without understanding the requirements though. People with strong opinions like to make up their own requirements without necessarily listening to the business, considering the team, considering the existing solutions, etc.

Re: PostgreSQL is enough (2024)

#58

Post by drdexebtjl »

mikeocool wrote: post #19

I love Postgres. I buy using it for many many things. I really don’t understand why everyone insists that you should use it as a work/message queue. There are lots of purpose-built bullet proof queuing systems that are simple to setup and administer (or just use SQS). Your queue is likely to have very different access patterns than the rest of your data, and sticking it in Postgres means you’re probably going to end…

Because you can’t two-phase commit between Postgres and the message broker, so you end up with a transactional outbox, which is already a queue.

Re: PostgreSQL is enough (2024)

#59

Post by nish__ »

dewey wrote: post #24
mikeocool wrote: post #19

I love Postgres. I buy using it for many many things. I really don’t understand why everyone insists that you should use it as a work/message queue. There are lots of purpose-built bullet proof queuing systems that are simple to setup and administer (or just use SQS). Your queue is likely to have very different access patterns than the rest of your data, and sticking it in Postgres means you’re probably going to end…

> There are lots of purpose-built bullet proof queuing systems that are simple to setup and administer (or just use SQS). Because in 99% of cases you don't need a purpose built solution (Even if engineers often think that) at the scale that most people operate in. Nothing is easier to setup and administer than the database you already use. We are using Postgres as a worker queue in production for many years, with mil…

> Nothing is easier to setup and administer than the database you already use.

What about the filesystem you already use? 99% of projects don't need a relational database at all.

Re: PostgreSQL is enough (2024)

#60

Post by AdieuToLogic »

LPisGood wrote: post #40
AdieuToLogic wrote: post #31

The first two linked resources: Simplify: move code into database functions Just Use Postgres for Everything Are disqualifying enough to not warrant further reading. A relational database is one form of persistent storage. They are great for managing persistent representations of key abstractions and their relationships. They are not application frameworks nor scalable messaging systems by design.

Not everything needs to “scale.” I think needing scale is relatively rare, actually.

> Not everything needs to “scale.” I think needing scale is relatively rare, actually.

The ability to scale a system is not limited to throughput metrics. If it where, I would agree with your point.

Scaling in complexity and/or ability to feasibly introduce significant change is also a desirable system attribute and the one which I predominantly had in mind. The likelihood of also being to scale from a throughput perspective is a nice bonus.

Post reply on HN