Interns with toasters: how I taught people about load balancers • Hacker News Board

Re: Interns with toasters: how I taught people about load balancers

#91

Post by rollcat » Sun, Apr 22, 2018, 8:11 PM UTC

jakobegger wrote: post #73

rollcat wrote: post #48

Earlier quoted context omitted.

Then you design your system to always respond as quickly as possible. HTTP even has a status code for that: 202 Accepted. You put the job in a queue and return some ID or cookie to the client, possibly also an ETA. Client can then poll for results (with bounded exponential back-off), again polling is just a quick key look-up; alternatively, if you have some means to push results to clients (like via WebSockets), use…

I'm not sure how that would solve the problem. Not every server is a front end web server. Think of something like video transcoding, or complex database queries. At some point you are going to have a number of servers that do the work, and a load balancer needs to balance the work between them. If one server is busy with a job that will take an hour, there's no point in randomly queueing up lots of work while others…

Read up on how task queues work - they do fill a role of a load balancer, your video transcoding case is a perfect example application. Basically workers can leave/join the pool at any time, poll for more work whenever idle/done, and signal failures so that jobs can be retried later. There's no such thing as an idle worker, unless the queue is empty. No "overloaded" workers either, running two CPU-bound tasks concurrently won't get either of them done sooner. Your monitoring / auto-scaling system can watch the length of the task queue and add/remove workers as needed. (You can also try to combine this with AWS spot instances.)

Re: Interns with toasters: how I taught people about load balancers

#92

Post by azernik » Sun, Apr 22, 2018, 8:41 PM UTC

randerson wrote: post #78

s_kilk wrote: post #74

Earlier quoted context omitted.

At work we have two endpoints on every service, `/status` and `/health-check`. The former is basically a "return true" endpoint, which can tell you if the service is alive and reachable. The latter will usually do something like "select 1;" from any attached databases and only succeed if everything is OK.

And the former is the one you want your Load Balancers to be checking. With a deep health check, even a brief database outage will cause every web server to be taken out of rotation, and then you're completely down for at least as many health check intervals as it takes for the LB to consider a host healthy again. Same goes for any other shared resource that is likely to affect all web servers if it becomes unavailab…

Or you just have smarter load balancers, that realize when all of their servers are acting funky and stop taking servers out of rotation.

Re: Interns with toasters: how I taught people about load balancers

#93

Post by sah2ed » Sun, Apr 22, 2018, 8:56 PM UTC

gaius wrote: post #87

pulkitsh1234 wrote: post #23

Maybe some people dismissed her problem as 'impossible' because she didn't inform what kind of load balancing technique was the load balancer using? I suspect what happened is that they didn't understand the problem, and so resorted to ineffective means to steer the attention away from their own inadequacies. ^ This statement is kinda harsh.

Every post on that blog is about how the author is incredibly clever and surrounded by dolts.

And I thought I was the only one that gets that vibe from the site's writing.

Anyway, several of the posts tend to do quite well on HN, so it looks like commenters look past that when they engage with the stories.

Re: Interns with toasters: how I taught people about load balancers

#94

Post by pitaj » Sun, Apr 22, 2018, 9:56 PM UTC

abakker wrote: post #71

snovv_crash wrote: post #39

This reminds me strongly of another system of distribution, which suffers from the same effect: differential gearboxes in cars. Since the torque to all the wheels is equal, if one wheel slips it very quickly takes all of the power of the engine, since power = rotational velocity * torque. Only if the speeds are similar is the allocation of power to the wheels similar. In cars, the solution is to make sure that the po…

There are a variety of mechanical systems used to combat this problem, not just thick grease. Limited Slip differentials can be employed to ensure that one wheel can only slip so far before the differential locks and then both wheels are forced to rotate together. LSDs can be employed in 3 positions front, rear, and center. The front and rear will lock left and right wheels together, the center one will lock the rota…

There's also the Torsen differential [1]

[1]: https://youtu.be/JEiSTzK-A2A

Re: Interns with toasters: how I taught people about load balancers

#95

Post by qooleot » Sun, Apr 22, 2018, 9:58 PM UTC

gaius wrote: post #86

randerson wrote: post #78

Earlier quoted context omitted.

And the former is the one you want your Load Balancers to be checking. With a deep health check, even a brief database outage will cause every web server to be taken out of rotation, and then you're completely down for at least as many health check intervals as it takes for the LB to consider a host healthy again. Same goes for any other shared resource that is likely to affect all web servers if it becomes unavailab…

Databases are 10-100x as reliable as the application tier in my experience

Although the OP didn't describe as such, but health checks that do a simple query are also testing the connectivity to the db. Someone might have hardcoded a db address in an environment variable, or there are connection pooling issues.

Re: Interns with toasters: how I taught people about load balancers

#96

Post by Terr_ » Sun, Apr 22, 2018, 10:03 PM UTC

azernik wrote: post #92

randerson wrote: post #78

Earlier quoted context omitted.

And the former is the one you want your Load Balancers to be checking. With a deep health check, even a brief database outage will cause every web server to be taken out of rotation, and then you're completely down for at least as many health check intervals as it takes for the LB to consider a host healthy again. Same goes for any other shared resource that is likely to affect all web servers if it becomes unavailab…

Or you just have smarter load balancers, that realize when all of their servers are acting funky and stop taking servers out of rotation.

Or to riff on this idea further: You make the concept of a node's "relative stability" part of the load-balancer logic the same way that "relative idleness" is a factor.

Then if all 100/100 nodes get taken down by some shared problem, the system simply degenerates into picking the idle-est of the 100.

Re: Interns with toasters: how I taught people about load balancers

#97

Post by sethammons » Sun, Apr 22, 2018, 11:30 PM UTC

randerson wrote: post #79

matt4077 wrote: post #35

Earlier quoted context omitted.

Random balancing only works with homogenous servers.

Isn't it almost always advisable to have homogenous servers in a web farm? What are some example cases there that doesn't make sense?

I'm on mobile, so won't look for the source. I heard Netflix benchmarks each new instance bootstrapped in AWS because they have found a dramatically high level of variability in performance from what should be home get out hardware. They toss out low performers and rebootstrap the instance.

Re: Interns with toasters: how I taught people about load balancers

#98

Post by Twirrim » Sun, Apr 22, 2018, 11:38 PM UTC

darshitpp wrote: post #38

> This is what happened when one bad web server decided it was going to fail all of its requests, and would do so while incurring the absolute minimum amount of load on itself. Good ELI5 explanation, but it doesn't really explain why the webserver failed the requests as it did. Or maybe I'm missing something?

Take your pick. There's a remarkable number of things that can cause a server to start returning 500s while the rest of the fleet is fine. Doesn't even have to be that specific server's fault (e.g. database behind it could have reached a connection limit handling the connection pools from all the other webservers, leaving this one in the dust. Fun part from this is it can result in the misbehaving server _moving_ as connections close and re-open)

Re: Interns with toasters: how I taught people about load balancers

#99

Post by Twirrim » Sun, Apr 22, 2018, 11:45 PM UTC

scopecreep wrote: post #7

I feel like if you have to teach them about load balancers as college interns, FB needs to find a better school to pull interns from.

Why would someone need to learn about load balancers in high school or earlier?

Also, don't mistake her teaching approach as indicating the people in the room weren't aware of what load balancers are or how they work. It's a good teaching technique to start out with some basic ground-work leading in to the point you wish to make. Starting out as she does ensures that everyone in the group knows _exactly_ what she's talking about before she gets to the key point, _and_ should be able to immediately understand what is going on and why.

Regardless of that, there's a lot of subtleties of dealing with load balancers that people rarely think of until they've been bitten by them. I've used load balancers quite regularly when interviewing candidates because I can almost always find some aspect of land balancers and load balancer behaviour that people aren't aware of. That gives me an ideal chance to explore a subject with a candidate and find out how quickly they can piece things together and learn.

Re: Interns with toasters: how I taught people about load balancers

#100

Post by zepolen » Mon, Apr 23, 2018, 12:57 AM UTC

randerson wrote: post #79

matt4077 wrote: post #35

Earlier quoted context omitted.

Random balancing only works with homogenous servers.

Isn't it almost always advisable to have homogenous servers in a web farm? What are some example cases there that doesn't make sense?

Containerization and spot instances.