i'm a visual person so i loved the animations! more like these please!
Load Balancing
161–170 of 243 posts
Re: Load Balancing
#162This is good teaching. You can get a concept across in a few minutes, or you can make someone struggle for hours to understand. I think classic academic teaching, leans towards the latter (imagine a latex pdf that introduces an alphabet of Greek letters), vs. this, which learns you the concept pretty quick. You can them do the Greeks letters version later.
Re: Load Balancing
#163This is so timely for me, I've been wondering how I can visualize load balancing algorithms for the last few weeks.
When trying to explain why round robin is bad, I often try to describe a situation where a backend cluster with a wide variance in performance can cause a cluster of clients using round-robin load balancing to start developing a sort of harmonic resonance where all the clients slowly synchronize on which servers they are hitting. However, it's hard to explain in words and a visual simulation would help me explain.
Do you think it would be easy for me to hack the source and add multiple clients to the simulation?
Re: Load Balancing
#164This is so timely for me, I've been wondering how I can visualize load balancing algorithms for the last few weeks. When trying to explain why round robin is bad, I often try to describe a situation where a backend cluster with a wide variance in performance can cause a cluster of clients using round-robin load balancing to start developing a sort of harmonic resonance where all the clients slowly synchronize on whic…
Shouldn't be too hard to add the clients since, in terms of the animation, they're just flipped versions of the server. The question is if this resonance is going to require some intentionality to it to demonstrate/replicate.
Re: Load Balancing
#165This is good teaching. You can get a concept across in a few minutes, or you can make someone struggle for hours to understand. I think classic academic teaching, leans towards the latter (imagine a latex pdf that introduces an alphabet of Greek letters), vs. this, which learns you the concept pretty quick. You can them do the Greeks letters version later.
(which teaches you the concept pretty quick.)
Re: Load Balancing
#166I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…
If I'm understanding the ask, you can certainly do this with Caddy. You could use the `forward_auth` directive to proxy to one upstream to authenticate the connection (by looking at the request headers contents) and possibly ask it to give you an upstream address as a response header, then you can use `reverse_proxy` to proxy the original request to that address from the auth request's header. You could also implement your own dynamic upstreams module (in Go) to do custom upstream selection logic. And Caddy has plenty of load balancing policy options to choose from (and a few more improvements in PRs I opened last week).
Re: Load Balancing
#167Several years ago, I implemented an application specific load balancer where the LB kept two queues. One was free_workers and the other was open_requests.
Initially, upon startup, each worker registers itself with the LB and gets added to the free_workers queue.
When requests arrive at the LB, the LB checks if there are free workers available. If yes, it dequeues a free worker and dispatches the request to that worker.
If no free workers are available, the LB adds the request to the open_requests queue.
When a worker finishes its work, it lets the LB know and the LB adds the free worker to the end of the free_workers queue and initiates another round of dispatching.
The parameter to watch for is the queue size for the open_requests queue.
(There were a few more nitty gritty details, but that was the concept at a high level)
Re: Load Balancing
#168What a great post!
Re: Load Balancing
#169A small note on "least connections" load balancing. The article says: Because the load balancer sits between the server and the user, it can accurately keep track of how many outstanding requests each server has. and this is the common case. But there are some cases, such as when serving audio/video streams (looking at you, porn :) where the server will send its response straight to the user, not back through the loa…
Yeah this was the most confusing part of the whole article to me, and seemed like the most important. I was like "how is it keeping track of how many connections are still being processed?"
In most situations (not all) the load balancer is actually a proxy. So when you type in example.com, the dns points to a load balancer. You connect to a load balancer, the load balancer opens a connection with the desired server (probably sitting in a private network somewhere), and proxies the traffic to the server.
You never actually connect to the server in this case. You stay connected to the load balancer the whole time, the load balancer makes a request to the server, which processes your request, and sends it back to the load balancer, which then proxies the data back to you in the response.
In this case the LB sits between you (the user) and the actual server running the application. In this case the LB knows how many connections each server has in the system because it is actually the one connected to the servers. If a server drops a connection, the LB knows because its the one that actually is losing connection. So it is fully aware of what all servers are doing in this scenario.
The only thing the load balancer might still need to do is ping servers for health checks to make sure they are online when the servers are not connected, this is also how it knows latency and other metrics.
Re: Load Balancing
#170If a picture is worth a thousand words.... a well done animation is gotta be at least 10K words. Thanks for making an old topic fun to read about again!
Thanks for your lovely words! <3
Great visual - very elegant! Although the denied requests look slightly wrong: perhaps quickly “bouncing” (less ease-in) and off the outside of the receivers would make it more understandable? But still a fantastic demo without any changes, bravo.