We've run into so many issues with PgBouncer and Postgres that I wish we didn't have to deal with as a new growing company. Nice to see more options out there!
PgDog is funded and coming to a database near you
241–250 of 275 posts
Re: PgDog is funded and coming to a database near you
#242We used `pgdog` as a proxy during a recent database backend migration (Heroku -> EC2 -> RDS) and it was much smoother than PgBouncer. Really nice seeing more things in this space, and having the team's work recognized.
Re: PgDog is funded and coming to a database near you
#243Earlier quoted context omitted.
Dynamo is a fundamentally different DB to Postgres. If your problem fits into the dynamo approach (I'd argue that more problems do), then you should be using it. No all problems fit, though.
Agreed, my critique was about how the article frames scalability. I've yet to see an OLTP problem that can't live in something like Dynamo. KV can model anything if you put in the work, the question is how much modeling discipline you trade for the scale, and in my experience the up front work is always worth it. Most of the time operational issues are swept under the rug and not consider tech debt. Take for example…
Upfront modeling work is always worth it, but that only holds if you actually know your access patterns upfront. Most teams don’t, especially early on.
Re: PgDog is funded and coming to a database near you
#244Earlier quoted context omitted.
It was one of the top real estate portals in the world. A lot of geolocation searches. New search every time someone moves the map. A ton of data sent to the client. Analytics in every page view. No clue how a shopping cart or checkout flow would drastically increase database load. It should just be basic CRUD. Building a shopping cart is something every student makes. Pages in a web store can be cached relatively ea…
You don't see how adding functionality that requires writing to the database rather than just reading from a cache could "drastically increase database load"?
Even if it's writing to the DB, I doubt basic queries like adding an item to a cart is going to need DB sharding.
I think one piece that someone else mentioned could require DB sharding and that is all the live data needed for tracking deliveries.
The actual website/app should not need more than one beefy Postgres instance.
Re: PgDog is funded and coming to a database near you
#245I have mentioned this before, but here it goes again: I'm really happy that there's more options for Postgres sharding and I applaud Pgdog and the team's efforts and energy. Having said that, this makes it a no-go for me: > shard_number = hash(data) % num_shards https://docs.pgdog.dev/features/sharding/basics/#terminology Most sharding solutions distribute the hash value over linear ranges, that then split across "vi…
Yeah good callout. We'll add rendezvous soon enough. Until then, being compatible with Postgres partitions has been advantageous -- while we build everything out, people were able to migrate to PgDog for the query routing layer while doing the resharding in Postgres.
Adding a sharding function in our architecture is relatively straightforward. We also support plugins which can control the flow (and direction) for queries, so our users can add their own (and they do!).
Re: PgDog is funded and coming to a database near you
#246Earlier quoted context omitted.
fwiw, we support cross-shard transactions. They are not magic though, just good old 2pc and a bit of coordination.
2pc is only safe if every part of the system has guaranteed uptime, which it never does. Assume that cross-shard transactions only work in the happy case and may result in inconsistent data otherwise. They also reduce the benefit of sharding, possibly down to worse performance than a non-sharded DB.
For sure. They should be used for "metadata"-style tables only. High throuput writes should be direct-to-shard.
Re: PgDog is funded and coming to a database near you
#247Congrats on the funding Lev! Just to say we're happy pgdog users here! One feature we quite like (of the proxy) is the handling of different connection settings per connection (i.e. statement_timeout). When we investigated RDS proxy (ages ago) it wasn't supported, I think the same was true for pgbouncer so it required a bunch of application changes. With pgdog, it just works transparently.
Thanks! Glad we made it relatively easy to migrate!
Re: PgDog is funded and coming to a database near you
#248Earlier quoted context omitted.
Doesn’t PgDog also handle the sharding by proxying the writes? Maybe I missed something but I thought this is their value prop. It’s not just another PgBouncer.
From the docs you have to tell it which shard to access - it doesn't automagically rewrite your statements.
We also do that! But it's not well documented at the moment.
Re: PgDog is funded and coming to a database near you
#249We used `pgdog` as a proxy during a recent database backend migration (Heroku -> EC2 -> RDS) and it was much smoother than PgBouncer. Really nice seeing more things in this space, and having the team's work recognized.
Awesome, glad it worked!
Re: PgDog is funded and coming to a database near you
#250I'd love to advocate for PgDog if there were more than 2 managed service providers. Adding a single company with no substitute in your supply chain feels hard
I didn't realize there are _any_ managed providers of PgDog out there...do tell!