Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PgDog is funded and coming to a database near you

pgdog.dev

Post reply on HN
Print view View original

131–140 of 275 posts

Re: PgDog is funded and coming to a database near you

#131

Post by true_religion »

tomtomtom777 wrote: post #125
aurareturn wrote: post #116

Earlier quoted context omitted.

Is 100k order per minute a lot? Even a single Postgres instance should serve that fine?

100k(s) orders per minute is several orders of magnitude more than realistic. Amazon does 20k orders per minute. Instacart doesn't need "100,000s of grocery delivery orders per minute". There must be some 0s added for the sake of the story.

According their 2026 Q1 filing they do about 90 million orders per quarter which is about 12 orders per second, 720 orders per minute.

It might make 100k row level changes per minute, but that’s a different metric.

https://www.sec.gov/Archives/edgar/data/1579091/000157909126...

Re: PgDog is funded and coming to a database near you

#132

Post by dotancohen »

tomtomtom777 wrote: post #125
aurareturn wrote: post #116

Earlier quoted context omitted.

Is 100k order per minute a lot? Even a single Postgres instance should serve that fine?

100k(s) orders per minute is several orders of magnitude more than realistic. Amazon does 20k orders per minute. Instacart doesn't need "100,000s of grocery delivery orders per minute". There must be some 0s added for the sake of the story.

Amazon does 20k peak, or 20k average? Website visitor peaks could easily be two orders of magnitude higher traffic than average for a few minutes.

Re: PgDog is funded and coming to a database near you

#133

Post by r7n »

levkk wrote: post #102
eikenberry wrote: post #99

> The reason DBs like Mongo or Dynamo exist is because Postgres has a scaling problem. I've used Postgres at a few places and the #1 problem was always high availability, not scaling. One Postgres cluster could easily handle 100000 transactions per minute, but when a primary node went down it was a page and manually failing over to the spare then manually replacing the spare. The manual tooling was very finicky but a…

Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/ Load balancer with health checks and failover, works out of the box. :) Battle-tested at this point too, so could be worth a look.

I've extensively used Dynamo (internally at Amazon and externally) and even founded a DB startup with it at it's core. Boiling down scalability of Postgres vs Dynamo as it's written in blog is a bit terse. Dynamo scales writes horizontally with the keyspace, forever. Postgres simply can't, and no number of layers between the machines and the developer changes that. Sharding, pooling, Citus are all layered on top of an engine where a given row's writes still land on one primary.

Re: PgDog is funded and coming to a database near you

#134

Post by dzonga »

bourbonproof wrote: post #19

the reason mongo is a joy to use in scaled env is because no additional setup/software needed and all drivers natively support secondary/primary writes/reads and topological changes. so it's end to end, and adding is as a new proxy in frontend of postgres leads to all clients being incompatible or the code itself has no control anymore about when to use a secondary and what allowed stall is acceptable for a particula…

once mongo rewrote their engine - it's performant, scales & easy to run. seems a lot of devs got burnt by the early issues don't consider it all together.

its probably the easiest database to run at scale. run & forget. you just have to do a little more work on the data modeling part before you write your application i.e consider your query patterns.

Re: PgDog is funded and coming to a database near you

#135

Post by nikolatt »

doctorpangloss wrote: post #121
gchamonlive wrote: post #110

Earlier quoted context omitted.

Not by itself if it's naive, but if it's able to assess target health and avoid degraded instances then it becomes a component in HA, the other being integrating an orchestrator for gracious recovery.

from their docs: > PgDog does not detect primary failure and will not call pg_promote(). It is expected that the databases are managed externally by another tool, like Patroni or AWS RDS, which handle replica promotion.

Why the snark comment? The PgDog project has been around for a while, it's not vibe coded.

Re: PgDog is funded and coming to a database near you

#136

Post by nijave »

parthdesai wrote: post #119
eikenberry wrote: post #99

> The reason DBs like Mongo or Dynamo exist is because Postgres has a scaling problem. I've used Postgres at a few places and the #1 problem was always high availability, not scaling. One Postgres cluster could easily handle 100000 transactions per minute, but when a primary node went down it was a page and manually failing over to the spare then manually replacing the spare. The manual tooling was very finicky but a…

Patroni 1.0 was released in 2016, i.e ~10 years ago. https://github.com/patroni/patroni

Yup Patroni handles automatic failures and cluster management quite well

Re: PgDog is funded and coming to a database near you

#137

Post by zamalek »

r7n wrote: post #133
levkk wrote: post #102

Earlier quoted context omitted.

Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/ Load balancer with health checks and failover, works out of the box. :) Battle-tested at this point too, so could be worth a look.

I've extensively used Dynamo (internally at Amazon and externally) and even founded a DB startup with it at it's core. Boiling down scalability of Postgres vs Dynamo as it's written in blog is a bit terse. Dynamo scales writes horizontally with the keyspace, forever. Postgres simply can't, and no number of layers between the machines and the developer changes that. Sharding, pooling, Citus are all layered on top of a…

Dynamo is a fundamentally different DB to Postgres. If your problem fits into the dynamo approach (I'd argue that more problems do), then you should be using it. No all problems fit, though.

Re: PgDog is funded and coming to a database near you

#138

Post by jsw »

r7n wrote: post #133
levkk wrote: post #102

Earlier quoted context omitted.

Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/ Load balancer with health checks and failover, works out of the box. :) Battle-tested at this point too, so could be worth a look.

I've extensively used Dynamo (internally at Amazon and externally) and even founded a DB startup with it at it's core. Boiling down scalability of Postgres vs Dynamo as it's written in blog is a bit terse. Dynamo scales writes horizontally with the keyspace, forever. Postgres simply can't, and no number of layers between the machines and the developer changes that. Sharding, pooling, Citus are all layered on top of a…

Curious how the DB startup with Dynamo at its core went. We use it heavily. The primary tricky thing for us at the moment is aligning pricing with workload value.

Re: PgDog is funded and coming to a database near you

#139

Post by r7n »

zamalek wrote: post #137
r7n wrote: post #133

Earlier quoted context omitted.

I've extensively used Dynamo (internally at Amazon and externally) and even founded a DB startup with it at it's core. Boiling down scalability of Postgres vs Dynamo as it's written in blog is a bit terse. Dynamo scales writes horizontally with the keyspace, forever. Postgres simply can't, and no number of layers between the machines and the developer changes that. Sharding, pooling, Citus are all layered on top of a…

Dynamo is a fundamentally different DB to Postgres. If your problem fits into the dynamo approach (I'd argue that more problems do), then you should be using it. No all problems fit, though.

Agreed, my critique was about how the article frames scalability. I've yet to see an OLTP problem that can't live in something like Dynamo. KV can model anything if you put in the work, the question is how much modeling discipline you trade for the scale, and in my experience the up front work is always worth it. Most of the time operational issues are swept under the rug and not consider tech debt.

Take for example AuroraDB: the sheer engineering it took to make SQL do scalable OLTP at all tells you how much that flexibility actually costs to keep.

Re: PgDog is funded and coming to a database near you

#140

Post by r7n »

jsw wrote: post #138
r7n wrote: post #133

Earlier quoted context omitted.

I've extensively used Dynamo (internally at Amazon and externally) and even founded a DB startup with it at it's core. Boiling down scalability of Postgres vs Dynamo as it's written in blog is a bit terse. Dynamo scales writes horizontally with the keyspace, forever. Postgres simply can't, and no number of layers between the machines and the developer changes that. Sharding, pooling, Citus are all layered on top of a…

Curious how the DB startup with Dynamo at its core went. We use it heavily. The primary tricky thing for us at the moment is aligning pricing with workload value.

We obsessed over optimizations and pushing the apis to the limits of how we could pack it.

So much so, we re-wrote the DynamoSDK to squeeze out more optimizations so we could be the same cost even though we were a layer in front of dynamo. We used key encoding and other various technique as well as managed capacity (on demand vs reserved) to transparently optimize workloads for price. In our experience we saw dramatic gains vs just vanilla SDK usage.

If you're curious, here was the marketing website, but we're now part of Databricks: https://stately.cloud/

Post reply on HN