Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Load Balancing

samwho.dev

Post reply on HN
Print view View original

171–180 of 243 posts

Re: Load Balancing

#171

Post by mlhpdx »

AYBABTME wrote: post #95

I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics. But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.

Indeed, and not only with HTTP. Stateless UDP protocols are the poster child for bad load balancing as a result of technology “tricking down” from HTTP. Push makes even more sense for those protocols.

Re: Load Balancing

#173

Post by mgoetzke »

photochemsyn wrote: post #29

I'm going to guess the style of this excellent effort was inspired in part by this popular HN poster? Creating a post in this manner with animations does take a lot of work but it's very effective and instructional: https://ciechanow.ski/gps/ Incidentally load balancing and dynamic routing in satellite networks looks like a complicated problem and is a subject of active research: https://www.ncbi.nlm.nih.gov/pmc/arti…

That page was indeed a masterpiece in visual communication.

Math and Music curriculum teachers and educators should spend time on making all topics approachable like this. It would be a great way to supplement learning by making it interactive. Instead of using iPads as glorified PDF readers.

Sadly the quality of e.g our Math Books (in Germany) is so low in many cases that I do not see this happening ever.

Re: Load Balancing

#174

Post by majormajor »

AYBABTME wrote: post #95

I feel like something missing from people's perspective when thinking of load balancing is to consider pull models. All push load balancing algorithms try to somehow predict how busy downstreams are. Some even go as far as having downstreams send back some utilization metrics. But if you use a pull-based approach, this is all sort of moot. Downstreams will pull work when they're ready for it.

This seems like it would be more resource demanding on the load balancer to manage that queue? If that's a single queue it's global state... if you partition it, well, that's just push-based load balancing moved inside your "outer" load balancer?

The latency of round-tripping "finished a request, give me another" is probably also best-case ever so slightly worse than "finished one of my requests, loading the next from the buffer" (the HTTP load balanced apps I've worked on have had request queues on the workers too (framework-level, usually)).

Moreso if you had multiple load balancers involved...

Re: Load Balancing

#175

Post by mgoetzke »

throwaway71271 wrote: post #8

we are so stuck with this push request load balancing its crazy, if we just switch to pull instead of push things get much smoother, and resources get better utilized you cant reliably guess if the instance where you will push your request actually has capacity to handle it, even using ML to guess it will still have thrashing properties but if you just let instances pull work, things work out for themselves sadly, th…

Peter Hintjens wrote a lot of different protocols in https://zguide.zeromq.org/docs/chapter4/#Service-Oriented-Re... and they mostly are part of the spec https://rfc.zeromq.org/spec/18/ now.

Some of these protocols are a good improvement when looking for the ultimate in speed and I used them where applicable, but as a first order approximation a simple load balancer approach is quite good enough. Especially when you do not have dedicated people vested in that domain for maintenance.

Re: Load Balancing

#176

Post by quickthrower2 »

zamnos wrote: post #165
quickthrower2 wrote: post #162

This is good teaching. You can get a concept across in a few minutes, or you can make someone struggle for hours to understand. I think classic academic teaching, leans towards the latter (imagine a latex pdf that introduces an alphabet of Greek letters), vs. this, which learns you the concept pretty quick. You can them do the Greeks letters version later.

(which teaches you the concept pretty quick.)

Thanks, it was a actually subtle reference/nod to the “learn you a haskell” book.

Re: Load Balancing

#177

Post by potamic »

throwaway71271 wrote: post #21
gbuk2013 wrote: post #19

Earlier quoted context omitted.

Moving to async model adds new set of operational challenges as well as some interesting failure scenarios. (Edit) Also, in practice you would need at least one more system to enqueue request into the broker, as the latter would typically not be exposed to the outside world. Request/response on the other hand is much simpler to configure and operate.

this is not async, its sync queue the lb puts a request where it has some reply_to (ip:port) where it waits (blockingly) for response from whoever picked up the request, it just does now know who that is until a reply comes

How do you ensure two backends don't pick up up the same request?

Re: Load Balancing

#178

Post by samwho »

dtjohnnymonkey wrote: post #163

This is so timely for me, I've been wondering how I can visualize load balancing algorithms for the last few weeks. When trying to explain why round robin is bad, I often try to describe a situation where a backend cluster with a wide variance in performance can cause a cluster of clients using round-robin load balancing to start developing a sort of harmonic resonance where all the clients slowly synchronize on whic…

You have my blessing to try. The source is all in one file and is just plain JS, no minification or anything.

I’ll warn you: the code is a bit messy. I wrote it knowing it was just for me and was a one off :D

Re: Load Balancing

#179

Post by samwho »

hamilyon2 wrote: post #156

I always thought that for hundreds of backends and above 10K RPS those algorithms scale poorly. First of all, scaling beyond single load balancer becomes a problem. The cost of updating counters becomes significant. If responses are quick, the cost additional bookkeeping adds doesn't pay off. Of you use weighted random all these problems go away. Is my knowledge outdated?

Weighted random is one of the algorithms available in the playground at the end. I did originally have a section talking about it but cut it for length reasons.

Very few people in practice are serving 10K RPS.

Re: Load Balancing

#180

Post by samwho »

hahnchen wrote: post #172

does a single load balancer ever get overwhelmed? what do you do when that happens?

Yeah, it does! There are a few options, I’ll mention 2:

1. You can use DNS to return IP addresses to multiple load balancers. 2. You can have N load balancers that are themselves behind another, beefier load balancer.

Plenty of other magical things that have been done to solve this problem, but I think those 2 are quite common and flow naturally from the ideas the post covers.

Post reply on HN