Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Load balancing and its different types

wisdomgeek.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Load balancing and its different types

#21

Post by Terretta »

saranshk wrote: post #7
Terretta wrote: post #3

I like this intro! Good to see ‘random’ redefined as pick random two then assign to one with least connections, which works strictly better than either random or least connections alone. Misses a couple categories that may be relevant: least hops or best transit type network-mapped balancing to get to the ideal set of servers globally, as well as a technique that not-so-simply connects the user to the geography with…

Thank you. I am not sure how to introduce transit and the concept of hops in an introductory post without explaining in depth about the networking side of it. Maybe you could help me out with that?

I’d think it’s fine to be hand-wavy:

Load balancing isn’t just about server load or congestion, it’s also about network load and congestion. If a web page or video takes longer for a user to download, it ties up the server longer too.[1]

Load balancing algorithms can also consider network paths or round trip times between the user and a server to give users a faster web download or video stream. To do this, they may use information from network routing topology, such as how many “hops” or routers between the user and the server, or may even triangulate actual network performance by assessing measurements from multiple data centers and load balancing to the most responsive.

1. See “snoshy” comment on latency in these comments: https://news.ycombinator.com/item?id=25920284 — roughly, you aim to avoid queuing or connection creep, as you mentioned in the intro, and speed of opening, transmitting data over, then closing the connection, can make a huge difference.

Re: Load balancing and its different types

#22

Post by toast0 »

dragontamer wrote: post #12

Load balancing as a strategy is used in far more than just web-applications! This article only discusses web-based load balancing, which is absolutely important, but doesn't discuss supercomputer scheduling load-balancing. Its arguably a different subject... but the concept is the same. When you have 4000 nodes on a supercomputer, how do you distribute the problem such that all the nodes have something to do? Superco…

> But in the case of web-applications, there's probably no way to really predict the "cost of performance" before you start processing the service request (what if its a Facebook request to a really old photograph? Facebook may have to pull it out of long-term storage before it can service that request. There's no real way to know at the load-balancer whether a picture-request would be in the cache or not... at least, not before you process the request to begin with!)

I worked at Facebook, but didn't touch anything related to this; so this is all conjecture.

The load balancer can't (or shouldn't) predict the cost to service a request; but the thing that generated the url for the image could; and that prediction could be passed in the url for the balancer to act on.

If you really need to balance by performance, it's probably simpler and accurate enough to provide frequent load feedback to the balancer. As long as you have a lot of requests, simple things work pretty well.

Re: Load balancing and its different types

#24

Post by toast0 »

kvhdude wrote: post #20

i would have liked to see details of layer4 vs layer7 load balancing. The latter invovles terminating a tcp session and reinitiating one to backend.

Layer 4 load balancing can be a huge reduction in work for the load balancer; especially if it's in a Direct Server Return configuration, where the load balancer only sees incoming packets, and response packets go directly from the server.

The downside is you lose any ability to balance based on details of the application protocol, it requires some specific network setup, and it's hard to find a DSR load balancer in managed hosting or cloud. I'm not sure if there's off the shelf software to manage DSR either (the basic pieces are there in most firewalls, but management isn't)

Re: Load balancing and its different types

#25

Post by saranshk »

supernova87a wrote: post #23

Speaking of, seems like wisdomgeek.com needs some load balancing right now...

Ha, I know. It's a single VM instance right now. I have been thinking of migrating to Gatsby for quite some time now. This unexpected traffic and the server limitations give me a reason to get working on it!

Re: Load balancing and its different types

#26

Post by crazypython »

saranshk wrote: post #25
supernova87a wrote: post #23

Speaking of, seems like wisdomgeek.com needs some load balancing right now...

Ha, I know. It's a single VM instance right now. I have been thinking of migrating to Gatsby for quite some time now. This unexpected traffic and the server limitations give me a reason to get working on it!

Or you could serve a fully static site with vanilla JS.

Re: Load balancing and its different types

#27

Post by tyldum »

samprotas wrote: post #6
sparrc wrote: post #2

Decent summary but a little out-dated on DNS load balancing. Major cloud services like AWS support health/status checks through DNS these days: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/re... It's also trivial to get around the client caching issue, just set a low TTL. Perhaps in the olden days providers had stricter limits on the minimum TTL you can set, but these days you can set it practically as l…

Having executed several "no-downtime" cutovers between systems via DNS updates, I will warn you that a surprising number of clients never re-resolve DNS, so the TTL is effectively "forever" from their point of view. For the rare case of lift-and-shift-ing for a system upgrade I felt morally okay about eventually pulling the plug on them, but I'd hesitate to design a system that relied on well-behaved DNS clients if I…

Another gotcha would be UDP based services. Since it is packet oriented and not connection oriented, when should it re-resolve? Most will not until the application is restarted.

Re: Load balancing and its different types

#28

Post by saranshk »

crazypython wrote: post #26
saranshk wrote: post #25

Earlier quoted context omitted.

Ha, I know. It's a single VM instance right now. I have been thinking of migrating to Gatsby for quite some time now. This unexpected traffic and the server limitations give me a reason to get working on it!

Or you could serve a fully static site with vanilla JS.

vanilla JS might be too much to maintain. I have already started working on the Gatsby version and will try and accelerate that.

Re: Load balancing and its different types

#29

Post by xorcist »

samprotas wrote: post #6
sparrc wrote: post #2

Decent summary but a little out-dated on DNS load balancing. Major cloud services like AWS support health/status checks through DNS these days: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/re... It's also trivial to get around the client caching issue, just set a low TTL. Perhaps in the olden days providers had stricter limits on the minimum TTL you can set, but these days you can set it practically as l…

Having executed several "no-downtime" cutovers between systems via DNS updates, I will warn you that a surprising number of clients never re-resolve DNS, so the TTL is effectively "forever" from their point of view. For the rare case of lift-and-shift-ing for a system upgrade I felt morally okay about eventually pulling the plug on them, but I'd hesitate to design a system that relied on well-behaved DNS clients if I…

Moving by DNS change isn't usually that bad. The old system (load balancer) can proxy requests to the new system. Most clients will follow DNS but the laggers won't have too much trouble. Assuming the service already works behind a load balancer of course, that is usually not something than can be fork-lifted in.

Re: Load balancing and its different types

#30

Post by crazypython »

saranshk wrote: post #28
crazypython wrote: post #26

Earlier quoted context omitted.

Or you could serve a fully static site with vanilla JS.

vanilla JS might be too much to maintain. I have already started working on the Gatsby version and will try and accelerate that.

I mean generating your HTML on the server. If you use HTML as a semantic markup language, it works pretty well. Then, using Vanilla JS for the interactivity. I doubt it's "too much to maintain"- what is harder in Vanilla JS than Gatsby?

You can write much of your site in raw HTML. For example, your right panel– you can write it in raw HTML, annotate it with some tags. Make it an iframe, just use CSS to get rid of the borders, or an XMLHTTPRequest. As long as you make good use of CSS, you can have a few tags and it'll work.

Post reply on HN