On Sharding
tbray.org
On Sharding
1–10 of 44 posts
Re: On Sharding
#2Sounds more like load balancing than sharding.
Re: On Sharding
#3Sounds more like load balancing than sharding.
Yeah; usually sharding implies some form of state or data storage that's split up across machines. If the same state is held on multiple machines it's called "mirroring"; if different state is held on different machines (and there's some function to determine which machine you need to talk to), it's "sharding". Load balancing encompasses both of these strategies, as well as the trivial cases where you have either stateless computations or read-only state that can be replicated.
Re: On Sharding
#4Def worth clicking through to the shuffle sharding thread. Simple concept (and somewhat common in my experience) but I’ve never seen the analysis before.
Re: On Sharding
#5Def worth clicking through to the shuffle sharding thread. Simple concept (and somewhat common in my experience) but I’ve never seen the analysis before.
Took me a long time to find (couldn't contrast the link), here it is:
https://twitter.com/colmmacc/status/1034492056968736768
Re: On Sharding
#6Another good strategy for load balancing/sharding that always strikes me as simple but also devilishly cleaver is random pick two: https://brooker.co.za/blog/2012/01/17/two-random.html
Re: On Sharding
#7> Load-sensitivity is one “smart” approach. The idea is that you keep track of the load on each shard, and selectively route traffic to the lightly-loaded ones and away from the busy ones. Simplest thing is, if you have some sort of load metric, always pick the shard with the lowest value.
Gotta be super careful with this one. We did this at reddit and it bit us bad. The problem was as soon as the load on a machine went down it got pounded with new requests and the load shot up, but it takes a few seconds for the load number to react to all the new requests. So we saw really bad see-saw affect.
We had to add extra logic to mark how long a machine had beed at a certain load and also randomly send requests to slightly more loaded machines to keep things even.
The moral of the story here is make sure you pick a metric that reacts to the change in request rate as quickly as your request rate changes!
Re: On Sharding
#8> But the cache is a distraction. The performance you’re going to get will depend on your record sizes and update patterns and anyhow you probabl don’t care about the mean or median as much as the P99.
True your 99th percentile slowest requests won't hit the cache, and certainly that caching won't solve all your scaling difficulties.
However, keeping requests for commonly-needed data away from (say) a DB cluster decreases the load on it at a given level of throughput, and that can be good for P99, and (as the post notes) caching can specifically help with super-hot data which can cause problematic hotspots in some sharding strategies.
Obviously situations vary and there're limits, but a cache seems like a legit tool, not just a band-aid, for a decent number of situations.
Re: On Sharding
#9Sounds more like load balancing than sharding.
Load balancing is just a subset of sharding though. It's how you shard your incoming traffic. The same strategies generally apply to both, but you get more leeway with traffic if it's stateless.
Re: On Sharding
#10My favorite sharding/load balancing algorithm is Highest Random Weight, or Rendezvous hashing [0]. It has all the benefits of consistent key hashing without the hotspots, and it doesn't require any coordination between nodes.