Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PgDog is funded and coming to a database near you

pgdog.dev

Post reply on HN
Print view View original

41–50 of 275 posts

Re: PgDog is funded and coming to a database near you

#41

Post by tempest_ »

yabones wrote: post #30

I'm curious how this might help with our biggest downtime-causer with postgres, which is major version upgrades. Poolers do a great job for failover and load balancing, but we consistently need ~10-20 minutes of downtime once or twice a year to do upgrades. Logical replication between old->new versions could probably help, but it would still require flipping everything over to the new cluster without partial writes o…

We use logical replication and a pause / swap in pgbouncer for ~5s of paused (but not failed) writes.

This is for DBs that are ~1-1.5TB but doesnt have a huge amount of churn/qps

Effectively what is described here https://www.pgedge.com/blog/always-online-or-bust-zero-downt...

Re: PgDog is funded and coming to a database near you

#43

Post by tingletech »

GiorgioG wrote: post #42
Ozzie_osman wrote: post #31

We sharded over 20 TB that we know about. This is probably a typo, right? 20TB isn't that big. I would imagine they've sharded a lot more than that

For a vast majority of use cases 20TB is positively enormous.

that article seems to suggest 20TB total over the dozen deployments in prod.

Re: PgDog is funded and coming to a database near you

#44

Post by happyopossum »

GiorgioG wrote: post #42
Ozzie_osman wrote: post #31

We sharded over 20 TB that we know about. This is probably a typo, right? 20TB isn't that big. I would imagine they've sharded a lot more than that

For a vast majority of use cases 20TB is positively enormous.

Sure, but 20TB in “the only database you need” is mere hours or minutes worth of data for many workflows.

Re: PgDog is funded and coming to a database near you

#45

Post by rswail »

faangguyindia wrote: post #17

i am not using any tool like pgbouncer and have not run into any issues so far. Is it even required these days? Have you guys tested your setup without these connection poolers/multiplexers?

Each connection is a process on the server, that takes up both CPU and RAM, it will run out.

This solves the thousands of clients case for read in a way that is transparent to the clients.

Yes it's required at large scale, especially if you want to distribute reads or shard to a particular geographical area.

Re: PgDog is funded and coming to a database near you

#46

Post by returningfory2 »

GiorgioG wrote: post #42
Ozzie_osman wrote: post #31

We sharded over 20 TB that we know about. This is probably a typo, right? 20TB isn't that big. I would imagine they've sharded a lot more than that

For a vast majority of use cases 20TB is positively enormous.

This product is for Postgres deployments that are so large they need to be sharded. For these use cases, I think 20TB is about normal.

Re: PgDog is funded and coming to a database near you

#47

Post by tux3 »

yabones wrote: post #30

I'm curious how this might help with our biggest downtime-causer with postgres, which is major version upgrades. Poolers do a great job for failover and load balancing, but we consistently need ~10-20 minutes of downtime once or twice a year to do upgrades. Logical replication between old->new versions could probably help, but it would still require flipping everything over to the new cluster without partial writes o…

Logical replication is how this is typically done. If you have some infra-as-code setup, you create a new cluster with identical settings except for the major version, import the schema, start copying data from a read-replica running the old version, stop accepting writes from the old version (downtime starts), sync the sequence numbers, and point your services to the new cluster (downtime ends).

If you use something like CloudNativePG they automate parts of the process with cli tools and declarative syntax. Otherwise you take the time to figure it out by hand. It might sound complicated, but just practice on your staging DB, and if all goes well you do the same procedure in prod.

Edit: Apparently Postgres 19 has a patch for one-shot logical replication of sequences! https://www.depesz.com/2025/11/11/waiting-for-postgresql-19-...

Re: PgDog is funded and coming to a database near you

#48

Post by tomnipotent »

jeltz wrote: post #39
boxed wrote: post #33

Earlier quoted context omitted.

Seconded. Coming from MySQL this is a huge regression that makes Postgres look like something from the 80s. I still wonder why this isn't seen as the absolutely highest priority.

I have not ran MySQL for some years but it at least used to have exactly the same issue. Upgrading a database with MySQL can take a long time if you have many tables. The main difference is only really that PostgreSQL does it with a separate tool, pg_upgrade, while MySQL does it as part of the main binary. For both MySQL and PostgreSQL you will need to use some kind of logical upgrades if you want no downtime.

MySQL has advocated for decades spinning up a replica with the upgraded version, waiting for it to catch up to master before promoting it to the new master. You can do the same thing with Postgres.

Re: PgDog is funded and coming to a database near you

#50

Post by rswail »

pantulis wrote: post #37
999900000999 wrote: post #29

How are 3 developers going to QA this properly ?

How are 3 developers going to sell that to any company? Procurement will have a field day.

They have funding. That's what it will be for. I wish them well and appreciate that people are still doing FOSS.

As long as they don't get undercut by the equivalent of AWS https://aws.amazon.com/rds/proxy/ which is a managed pgbouncer.

Post reply on HN