I have mentioned this before, but here it goes again: I'm really happy that there's more options for Postgres sharding and I applaud Pgdog and the team's efforts and energy. Having said that, this makes it a no-go for me: > shard_number = hash(data) % num_shards https://docs.pgdog.dev/features/sharding/basics/#terminology Most sharding solutions distribute the hash value over linear ranges, that then split across "vi…
Yeah good callout. We'll add rendezvous soon enough. Until then, being compatible with Postgres partitions has been advantageous -- while we build everything out, people were able to migrate to PgDog for the query routing layer while doing the resharding in Postgres. Adding a sharding function in our architecture is relatively straightforward. We also support plugins which can control the flow (and direction) for que…
* Adding a sharding function, as you say.
* Developing an external service for metadata (shard placement) or alternatively have that metadata in one place and replicate (consistently!) to every query router.
* Implementing functions/catalogs for the users to understand the placement and configure/alter it.
* Implementing shard migration / rebalancing capabilities, possibly using Postgres logical replication (plus notable automation).
Here's one idea if you follow this path, something that Citus doesn't have: make the sharding function pluggable and pick one by default which is well-known and available in many languages (e.g. xxhash). If you do so, and guarantee stability of those functions, they could be used externally (applications) to route queries / inserts especially to the appropriate shard. While it makes application more complex, it may allow (combined with access to the metadata service) for faster ingestion paths (this is often known as application assisted sharding), and its not exclusive of the query routers.
Edit: formatting