Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL is enough (2024)

gist.github.com

Post reply on HN
Print view View original

61–70 of 97 posts

Re: PostgreSQL is enough (2024)

#61

Post by zxexz »

ronbenton wrote: post #44

I have worked some places where a significant amount of business logic lived in database functions and triggers and, hoo boy, that was really hard to reason about. If you're disciplined enough to have migrations around that implemented all that stuff, you're still going to have an unpleasant time piecing all of it together when debugging. But often you're in a state where you don't even have those breadcrumbs and you…

Yeah that's super frustrating. It sounds like a classic IT dev issue when IT is ops though to me - and I'm happy to be wrong.

I personally believe (notice the caveat here) that debugging relatively well designed databases on a good RDBMS is an easy task, if given access to logs, data (or even a redacted replica), and docs. Choose the docs along with one of the first two and it shouldnt remain a mystery for long. I've worked with postgresql a lot over the last couple decades, and even the most convoluted live-in-prod bug isn't wasn't half as bad as the vibe coded nonsense I see and debug weekly (not hating on AI; fwiw this is more a symptom of process/lack there of around integration of the tech IMO).

Re: PostgreSQL is enough (2024)

#62

Post by dewey »

nish__ wrote: post #59
dewey wrote: post #24

Earlier quoted context omitted.

> There are lots of purpose-built bullet proof queuing systems that are simple to setup and administer (or just use SQS). Because in 99% of cases you don't need a purpose built solution (Even if engineers often think that) at the scale that most people operate in. Nothing is easier to setup and administer than the database you already use. We are using Postgres as a worker queue in production for many years, with mil…

> Nothing is easier to setup and administer than the database you already use. What about the filesystem you already use? 99% of projects don't need a relational database at all.

Sqlite is basically your file system and comes with the benefit of being a relational database with all it's advantages.

Re: PostgreSQL is enough (2024)

#63

Post by zxexz »

brianwawok wrote: post #43
SwellJoe wrote: post #13

I'd go further, and say that most of the time, "SQLite is enough". But, yes, PostgreSQL is all I ever use for anything that needs to be big. I ported a big old web app that had ScyllaDB, Elastic Search, Redis, and probably some other stuff I've forgotten. It got PostgreSQL+PostGIS (it's a mapping app), that's it. I'm sure there's some situation where it would be worth looking at all that other stuff, but it's ridicul…

Redis is basically free and nothing like the other tools you mentioned. Anytime I need a quick cache that will survive reboots it’s a winner. Agree on the other stuff though.

Yeah. Redis (valkey now for me mostly) is where I go for that extra oomph if the patterns make sense (or some level of scripting for dev work makes sense; the lua integrations is chef's kiss.).

Re: PostgreSQL is enough (2024)

#65

Post by f3408fh »

rc2026 wrote: post #25

Postgres has its advantages, but for message queues I’d stick with SQS. I built out a trading firm last year that was basically Postgres, some dotnet services and SQS queues and we got acquired for $140M. You can build some fairly formidable systems if you keep them simple.

Sounds interesting. Can you elaborate? You got acquired one year into doing business for 100M+? Impressive

Re: PostgreSQL is enough (2024)

#66

Post by jghn »

thayne wrote: post #15
stuaxo wrote: post #10

Earlier quoted context omitted.

That's fine. There are plenty of projects that don't hit that scale.

There are also plenty of projects that do. PostgreSQL is good enough for most projects. But it isn't enough for every project.

The problem is that way too many people assume they're in the latter group, when the proportions are very much the other direction

Re: PostgreSQL is enough (2024)

#67

Post by cyberax »

leoqa wrote: post #38
cyberax wrote: post #16

Earlier quoted context omitted.

HA is now pretty good on Postgres. As for scale... Just use a larger machine. This works for regular transactional data until you're at something like Amazon scale. Edit: Think about this, suppose that you store 1 megabyte of data for each of your customers. So if you have a million customers, it's just 1Tb. And these days, you can have a server with 10Tb RAM delivered overnight. Although you might have to sell your…

128Gb of RAM is like 1.5k

For a non-ECC version. A single-module 128Gb ECC stick is about $4500. HBM is even more expensive.

Re: PostgreSQL is enough (2024)

#68

Post by mindwok »

fastball wrote: post #5

> But the bar should be high: only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative Why do I need to push Postgres to its limits before using a different solution? Throwing a hosted Redis in front of some hot-path API calls is very straightforward and easier to reason about than materialized views or UNLOGGED tables.

I haven't really used redis much so curious to hear your perspective - this seems the opposite to me? A materialised view is just taking the data I already have and rendering it in a different way to speed up my access patterns. It's easy for me to understand where its all coming from, and it's all directly mapped back to the source data so if things change I can easily understand why it might break etc.

For redis, it seems there's no "out of the box" way to take some data from my DB and cache it. It seems it needs to be hand rolled per query you're optimising, you lose any structural link to the source data (redis doesn't know about my table structure), and now I have another service I need to worry about. Or is it much easier nowadays than I am thinking?

Re: PostgreSQL is enough (2024)

#69

Post by mindwok »

fastball wrote: post #50
sgarland wrote: post #45

Earlier quoted context omitted.

Those examples are all equally difficult to reason about. Cache invalidation is equivalent to refreshing a materialized view, and UNLOGGED tables bring about new and exciting ways lose data.

Cache invalidation on Redis is setting a TTL of 60s on the kv I just set, in a single atomic operation.

You can also pg_cron refresh a materialised view every minute which seems similar.

Re: PostgreSQL is enough (2024)

#70

Post by esafak »

throwawayffffas wrote: post #12
bitlad wrote: post #6

PostgreSQL is good enough until it's not good enough, when you realize all the bad design decisions that were made before it hits scale. It is the decisions people make around not partitioning, HA, replication that makes it not good enough.

These are not bad decisions they are reasonable tradeoffs at the beginning. Requiring HA, partitioning, and replication are good problems to have. The alternative is spending engineering time on setting all these up for a failed service with like 100 users.

It is not good to have a problem that requires a year of migration by a dedicated team (typical case if you have this 'good problem') when you could have simply spent another day or two to architect it right. You should at least have a path to scale when the time comes.
Post reply on HN