Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Load Balancing

samwho.dev

Post reply on HN
Print view View original

231–240 of 243 posts

Re: Load Balancing

#231

Post by dragontamer »

gyourstory35 wrote: post #230

[dead]

Queuing theory is nice, but then again its completely abstract and mathematical. Real life issues get in the way.

In particular: for many parallelization problems, its simply not worth the effort to ever load balance. Tasks are too small and the "time spent calculating where to go" is more expensive than just accomplishing the task to begin with. (Ex: Matrix multiplication can be seen as a parallelization of multiplies, followed by a set of additions. The multiplies are too cheap to perform any form of load-balancing, executing in just one clock tick).

So in reality, we have "fine grained parallelism", and "coarse grained parallelism". Coarse grained is where load balancing is feasible (the task is "big enough" that its worth spending a bit of CPU time figuring out load balancing).

"Fine grained" is very difficult, and is in the realm of CPU design (CPUs will execute your instructions out-of-order to discover parallelism)... though software also plays a role.

----------

That being said, I probably should study more queuing theory myself. The math seems less about "how to design a good parallel system" and more focused on "how to measure a parallel system" (ex: throughput * latency == state... which means you can measure latency == state/throughput, and other such mathematical tricks).

The math is simple, albeit abstract. Its not there for "deeper understanding", its there for "even basic understanding" (but much much faster to calculate if you already know queuing theory).

Re: Load Balancing

#236

Post by preseinger »

throwway120385 wrote: post #123
zokier wrote: post #88

Earlier quoted context omitted.

That sort pull architecture is interesting thought. I do see it working best when a worker nodes ability to do work is very clear cut, i.e. the response time does not significantly vary with the number of concurrent requests it is handling. Here is an example where it would not work well: lets imagine workers are doing a CPU bound task and each worker has one cpu (for simplicity), but we have also identified that we…

With a pull architecture you wouldn't identify a request queue depth up front. Rather, each incoming connection gets delayed until a worker attaches to EG the TCP stream. If you're serving a webpage the client wouldn't see anything other than a little load lag, especially if your load balancer took care of the TLS connection before pausing to wait for a worker. So if you can handle every incoming request with one wor…

tcp is a layer well below what we're discussing here

"handling a request" is something that is defined at the application layer

Re: Load Balancing

#238

Post by samwho »

talentedRed wrote: post #187
samwho wrote: post #107

Earlier quoted context omitted.

I used https://pixijs.com/ . This was my first project with it and it was really nice to use. I fell in to a few traps here and there, but generally it wasn't too bad to get the results I wanted with it. :)

I'd love to see the source code of it :)

It’s just plain JS, you can see it here: https://samwho.dev/js/load-balancers.js

Just like the good old days :)

Re: Load Balancing

#239

Post by preseinger »

throwaway71271 wrote: post #8

we are so stuck with this push request load balancing its crazy, if we just switch to pull instead of push things get much smoother, and resources get better utilized you cant reliably guess if the instance where you will push your request actually has capacity to handle it, even using ML to guess it will still have thrashing properties but if you just let instances pull work, things work out for themselves sadly, th…

if your load balancer holds on to each received request until a downstream server happens to pull that request and process it, then it's not a load balancer, it's a queue

different concept, different semantics, different results

Re: Load Balancing

#240

Post by preseinger »

hamilyon2 wrote: post #156

I always thought that for hundreds of backends and above 10K RPS those algorithms scale poorly. First of all, scaling beyond single load balancer becomes a problem. The cost of updating counters becomes significant. If responses are quick, the cost additional bookkeeping adds doesn't pay off. Of you use weighted random all these problems go away. Is my knowledge outdated?

updating a counter (in memory on a single host) is typically a single- or double-digit-nanoseconds operation, basically never the bottleneck in anything
Post reply on HN