21–30 of 113 posts
A favorite resource: https://learn.microsoft.com/en-us/azure/architecture/pattern... Microsofts Azure Cloud Patterns is some of the best documentation out there. It's not Azure centric. It focuses on why you may want to do something and describes the techniques that are commonly used to solve it. It's pretty great.
Earlier quoted context omitted.
the only real answer here is to shard by customer optimistically, you can try to shard by read use case, but that's never gonna be stable over time if you need a true multi-tenant system you can only shard by individual entity and move all of the composition logic to the next layer up, there's no way to cheat
> the only real answer here is to shard by customer No. Pick a a stable, guaranteed-to-exist, shard key (composite or atomic properties) and use that. If composite, order properties used in most-to-least distinct value distribution. > if you need a true multi-tenant system you can only shard by individual entity and move all of the composition logic to the next layer up, there's no way to cheat This is incorrect. Sha…
This is a pretty risky approach since it's almost certainly the case that you won't end up evenly distributing your data across shards using this method.
Thanks for this article planetscale. I've been enamoured with sharding recently but more for multithreaded performance. I want multimaster postgres. I started trying to write a postgres synchronizer, by sorting every data by row and column and hashing the data of every column and row, then doing a rolling hash of the data. In theory, two databases can synchronize by sending the final hash of their data and then doing…
You need Citus
Recently they've been acquired by Microsoft & have started to been integrated into their CosmosDB offering. Anybody have good experience with this DBaaS offering?
One thing I notice is the over-usage of sharding, especially hash-based, might turn your Relational Database into just another key-value store, with consistency constraints moving into application code, and you lose many advantages of a traditional RDBMS
What's the alternative as things get too big? Sharding by date? By client? How do you prevent hotspots?
A favorite resource: https://learn.microsoft.com/en-us/azure/architecture/pattern... Microsofts Azure Cloud Patterns is some of the best documentation out there. It's not Azure centric. It focuses on why you may want to do something and describes the techniques that are commonly used to solve it. It's pretty great.
A favorite resource: https://learn.microsoft.com/en-us/azure/architecture/pattern... Microsofts Azure Cloud Patterns is some of the best documentation out there. It's not Azure centric. It focuses on why you may want to do something and describes the techniques that are commonly used to solve it. It's pretty great.
Just want to say thanks for pointing out this resource. Will make for some great long weekend reading!
Earlier quoted context omitted.
What's the alternative as things get too big? Sharding by date? By client? How do you prevent hotspots?
the only real answer here is to shard by customer optimistically, you can try to shard by read use case, but that's never gonna be stable over time if you need a true multi-tenant system you can only shard by individual entity and move all of the composition logic to the next layer up, there's no way to cheat
Sharding at table level is very complex and expensive. Fully give a single DB per tenant is very practical, and the reasons to do sharding (like reporting) is where you do the other fancy things (like ship events in to kafka, etc, etc).
Also, most issues of scalability are dominated by a few tenants that consume most resources, and distribute the loads is more easy per-tenant.
A favorite resource: https://learn.microsoft.com/en-us/azure/architecture/pattern... Microsofts Azure Cloud Patterns is some of the best documentation out there. It's not Azure centric. It focuses on why you may want to do something and describes the techniques that are commonly used to solve it. It's pretty great.