The visualizations are perfect, I will add this to our trainee introduction on loadbalancing.
Load Balancing
91–100 of 243 posts
Re: Load Balancing
#92Least connections is intuitively very sensible algorithm, but it has one pitfall: if backend server starts returning errors for whatever reason (e.g. load shedding) quicker than typical responses, it can lead to situation where disproportionate number of requests are directed to that one server. Also typically you'd have more than one load balancer instance, and at least naive least connections requires shared state…
There is a talk about Fitbit load balancer on YouTube, where their algorithm is described. They used modified least connections algorithm, which also tracks failure rate of each node, decreasing the traffic when error rate is increased. The video is called "A Frontend Server, Front to Back by Zach Tellman", timestamp 33:34.
Re: Load Balancing
#93[dead]
Re: Load Balancing
#94Earlier quoted context omitted.
A pull approach seems difficult to manage when you have many layers in your load balancing. In small setups, you may just have one layer with a single load balancer (well, hopefully at least a hot-warm pair), but larger setups often have multiple levels. There may be a network level traffic split to multiple frontend load balancers via something like ECMP; those frontends may connect directly to the origin hosts, or…
Nice to see someone mention direct server return or as BigIP called it nPath routing. This was an effective scaling method for handling small request that returned large payloads (audio and video files). I don't know how well known this configure is or whether it is still viable in an all TLS world.
It doesn't seem particularly well known. It works fine with TLS, but the origin servers need to do the TLS termination (IMHO, this is better for security than having your load balancers do it, but it does mean you have to work harder on key distribution). On a non-DSR load balancer, doing TLS termination on the origins means the load balancer has less application data to work with (request path, response status code, etc) in making load balancing decisions, but for DSR, the load balancer never had any of that, so adding TLS doesn't disadvantage the load balancer any more.
TLS session establishment is expensive, so why would I want my load balancers to do it anyway? :P
Re: Load Balancing
#95I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics.
But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.
Re: Load Balancing
#96Earlier quoted context omitted.
One of the nicest things about pull load balancing is that it completely replaces the need for a separate registration + health check system. Once you consider that, IDK if someone can really claim "more complicated."
Doesn't it just flip the registration/check need instead of eliminate it? You'd need to register the brokers/loadbalancers to the workers so they know where to pull work from?
Yes.
But the load balancers/brokers are usually far less dynamic. I've never seen a worker health check a RabbitMQ broker.
Re: Load Balancing
#97Earlier quoted context omitted.
One of the nicest things about pull load balancing is that it completely replaces the need for a separate registration + health check system. Once you consider that, IDK if someone can really claim "more complicated."
I honestly had never thought hard about reversing the relationship and having workers pull. This point about no longer requiring health checks is a real "woah" moment. Thanks for expanding my mind!
You still need health checks, though? Otherwise, how do you tell the difference between: "no traffic + server alive" vs "some traffic + server is dead". Yeah, you can monitor throughput on a load balancer, but if I ever again wake up from an alert about not traffic being served - I will throw hands.
Re: Load Balancing
#98I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics. But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.
For most protocols like HTTP there's no software to do this and it's not worth developing something nonstandard.
Re: Load Balancing
#99I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics. But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.
The pull model works beautifully for background work queues. For load balancing it seems it would take a bit of re-plumbing on the expectations of request/response. How would you envision this working with something like HTTP? It seems it would reverse the direction, which would be interesting.
Re: Load Balancing
#100I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics. But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.
The pull model works beautifully for background work queues. For load balancing it seems it would take a bit of re-plumbing on the expectations of request/response. How would you envision this working with something like HTTP? It seems it would reverse the direction, which would be interesting.
Very similar to background queues but the load balancer holds the request and dispatches it to a waiting worker, or waits until someone comes to serve it. While the worker handles the request, the load balancer proxies.
The request serving is the same, the way the work is dispatched is just inverted.