Load Balancing • Hacker News Board

Re: Load Balancing

#211

Post by preseinger » Wed, Apr 19, 2023, 4:29 PM UTC

0x457 wrote: post #97

samwho wrote: post #40

Earlier quoted context omitted.

I honestly had never thought hard about reversing the relationship and having workers pull. This point about no longer requiring health checks is a real "woah" moment. Thanks for expanding my mind!

You still need health checks, though? Otherwise, how do you tell the difference between: "no traffic + server alive" vs "some traffic + server is dead". Yeah, you can monitor throughput on a load balancer, but if I ever again wake up from an alert about not traffic being served - I will throw hands.

server health is necessarily a function of the actual production traffic it receives, determined at the application layer, as observed by a specific observer

it can't be known by the server itself, as (among many other reasons) the server can't know about network issues between itself and any upstream caller

it can't be determined by out-of-band health check queries, because those queries don't represent actual traffic, the simplifying assumption that they _do_ introduces many common failure modes that any seasoned engineer can speak at length about

health checks can be a nice additional signal on top of monitoring actual prod traffic, but they can't be used by themselves, they just don't capture enough relevant information

Re: Load Balancing

#212

Post by preseinger » Wed, Apr 19, 2023, 4:33 PM UTC

adql wrote: post #194

throwaway894345 wrote: post #78

Earlier quoted context omitted.

Agreed on the “whoa moment”. Not needing health checks seems pretty compelling. I’m curious if there are any off-the-shelf pull-based load balancer that don’t require the full HTTP request copied onto the queue before handing off to a worker?

You need health checks. If not for LB then for your own monitoring.

you don't

a server can return 200 OK to every health check query and 5xx to every production request, is that server up or down? hopefully clear it is down

similarly, if it returns 5xx to every health check query but 200 to every prod request, is it up or down? hopefully clear it is up

if you want to monitor the status of a server (or application) for internal purposes, that's fine, but that's (at best) supplementary signal for the decisions made by a load balancer, not something you can make load balancing decisions on in isolation

Re: Load Balancing

#213

Post by preseinger » Wed, Apr 19, 2023, 4:43 PM UTC

jayd16 wrote: post #76

throwaway71271 wrote: post #21

Earlier quoted context omitted.

this is not async, its sync queue the lb puts a request where it has some reply_to (ip:port) where it waits (blockingly) for response from whoever picked up the request, it just does now know who that is until a reply comes

If we're not talking about an async model then the suggestion is much less drastic than it sounded at first. In that case the crux of your desire is simply allowing the hosts to signal readiness more directly. You would almost never actually wait for host machines to dial in. You would have a list of hosts that are ready or not ready as they would almost always be ready for more. You want to assume readiness (as this…

it is not possible for a remote destination host to signal to a sending host that it is ready, or not ready, for more requests, in a reliable way

readiness is not knowable by a receiver, it is a function of many variables, some of which are only knowable to a sender, one obvious example is a network fault between sender and receiver, there are many more

even the concept of "load" reported by a receiving application isn't particularly relevant, what matters is the latency (and other) properties of requests sent to that application as observed by the sender

health is fundamentally a property that is relative to each sender, not something that is objective for a given receiver