I was explaining the DNS system and load balancing today and I kind of mixed it all up based on this wonderful link. Thanks, I will share this out with that person to undo the damage I might have done.
Load balancing and its different types
11–20 of 30 posts
Re: Load balancing and its different types
#12Load balancing as a strategy is used in far more than just web-applications!
This article only discusses web-based load balancing, which is absolutely important, but doesn't discuss supercomputer scheduling load-balancing. Its arguably a different subject... but the concept is the same.
When you have 4000 nodes on a supercomputer, how do you distribute the problem such that all the nodes have something to do? Supercomputer problems are sometimes predictable (ie: matrix-multiplications), and you can sometimes "perfectly load balance without communication".
But in the case of web-applications, there's probably no way to really predict the "cost of performance" before you start processing the service request (what if its a Facebook request to a really old photograph? Facebook may have to pull it out of long-term storage before it can service that request. There's no real way to know at the load-balancer whether a picture-request would be in the cache or not... at least, not before you process the request to begin with!)
-----------
In any case, I think adding "Predict the computational cost, calculate the costs you distributed to different nodes, and then distribute the new load to the node with lowest computational cost given so far" is a good method that works in some applications. (All blocks in a dense matrix multiplication have the same cost, so just keep passing out subblocks to all nodes as you're working on the problem)
Re: Load balancing and its different types
#13I was explaining the DNS system and load balancing today and I kind of mixed it all up based on this wonderful link. Thanks, I will share this out with that person to undo the damage I might have done.
We all learn everyday. I am learning from the comments here as well. The best we can do is accept we were wrong and correct it.
Re: Load balancing and its different types
#14Load balancing as a strategy is used in far more than just web-applications! This article only discusses web-based load balancing, which is absolutely important, but doesn't discuss supercomputer scheduling load-balancing. Its arguably a different subject... but the concept is the same. When you have 4000 nodes on a supercomputer, how do you distribute the problem such that all the nodes have something to do? Superco…
Concepts are definitely transferrable across domains. And then adapted according to desired outputs. Cost of performance is interesting and somewhat similar to the least-time approach but adapted to the supercomputers domain.
Re: Load balancing and its different types
#15Load balancing as a strategy is used in far more than just web-applications! This article only discusses web-based load balancing, which is absolutely important, but doesn't discuss supercomputer scheduling load-balancing. Its arguably a different subject... but the concept is the same. When you have 4000 nodes on a supercomputer, how do you distribute the problem such that all the nodes have something to do? Superco…
Arguably, one of the most important characteristics of a load balancer is to have extremely low latency. If you're balancing loads, you want to be very quick about making a decision. When generating predictions about the computational cost, that can itself add in a computational cost that might result in a non-negligible amount of computational cost overhead.
Inherently, the idea that you're talking about boils down to having a way to characterize the nature of the request flows in such a manner that they can be evenly distributed. The ideal way to characterize them then, would be to know this information beforehand such that it does not require any computation at all to normalize the costs. As such, the best strategy would be to actually segregate traffic flows such that they're forwarded to "dumb" load balancers that use one of the strategies from TFA like weighted round robin.
Of course, there are many such optimizations available, but TFA seems to be targeting a beginner level introduction to a rather complex topic. As you describe, load balancing and scheduling algorithms have a pretty high overlap in terms of their theoretical foundations, and these concepts manifest themselves throughout any large scale system.
Re: Load balancing and its different types
#16Decent summary but a little out-dated on DNS load balancing. Major cloud services like AWS support health/status checks through DNS these days: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/re... It's also trivial to get around the client caching issue, just set a low TTL. Perhaps in the olden days providers had stricter limits on the minimum TTL you can set, but these days you can set it practically as l…
TTL is difficult in practice due to client implementations and other issues like that. Be careful using DNS anything. DNS was not designed to immediately resolve anything. That's why IPs are mostly used.
Re: Load balancing and its different types
#17Nice succinct writeup. Here's a great deep dive on "Practical Load Balancing with Consistent Hashing" from a Vimeo engineer (2017) [0].
Re: Load balancing and its different types
#18Decent summary but a little out-dated on DNS load balancing. Major cloud services like AWS support health/status checks through DNS these days: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/re... It's also trivial to get around the client caching issue, just set a low TTL. Perhaps in the olden days providers had stricter limits on the minimum TTL you can set, but these days you can set it practically as l…
Many applications do not refresh their DNS with every connection either. Take for example an Apache reverse proxy that's reusing long lived connections. So updating DNS may still require restarting/reloading many upstream services.
https://stackoverflow.com/questions/52032150/apache-force-dn...
Re: Load balancing and its different types
#19Software load balancing solutions have now more algorithms such as least time (nginx+ for example). And yes some ISP cache DNS entries for a long time... But DNS load balancing should be used only on disaster scenarios to mitigate.
DNS load balancing works good enough if you have smart enough clients (not web browsers), and your pool of server IPs is fairly static. If you can select randomly from a list of names, and then try several of the A/AAAA records from that result, then you may have some delay if you pull a dead server from a cached record, but it won't be too bad. SRV records and really smart clients should work pretty well too, but not a lot of people have really smart clients.
The vast majority of ISP caches won't keep your low TTL records in cache for years, but some do; this is a problem if you have to move your load balancers ever too though.
Depends on how stable your servers are vs your load balancers, and how many connections you need; and if you have enough IP addresses to give public IPs to your servers. Also, if you really absolutely need to control the load precisely, DNS isn't going to ever give you that.
Re: Load balancing and its different types
#20i would have liked to see details of layer4 vs layer7 load balancing. The latter invovles terminating a tcp session and reinitiating one to backend.