Interns with toasters: how I taught people about load balancers • Hacker News Board

Re: Interns with toasters: how I taught people about load balancers

#31

Post by nvarsj » Sun, Apr 22, 2018, 11:24 AM UTC

BillinghamJ wrote: post #21

captn3m0 wrote: post #14

Earlier quoted context omitted.

AWS ELBs work this way. They offer more requests to clients with the shortest backlog

Do you have a link to any documentation mentioning this? As far as I’m aware - using ELBs heavily day to day - it doesn’t maintain a backlog and simply distributes requests on a round robin basis the moment they come in. There is no queue of waiting requests I don’t believe. If the request can’t be fulfilled immediately, it is rejected.

From the AWS docs:

> With Classic Load Balancers, the load balancer node that receives the request selects a registered instance using the round robin routing algorithm for TCP listeners and the least outstanding requests routing algorithm for HTTP and HTTPS listeners.

Re: Interns with toasters: how I taught people about load balancers

#32

Post by andrew_wc_brown » Sun, Apr 22, 2018, 11:36 AM UTC

Based on how verbose this article is I think maybe her original article was misunderstood because it was a chore to sort out the junk from the content just like this one.

Too much flourishing, poor pacing

Re: Interns with toasters: how I taught people about load balancers

#33

Post by rfrey » Sun, Apr 22, 2018, 11:42 AM UTC

BillinghamJ wrote: post #12

rajacombinator wrote: post #8

Earlier quoted context omitted.

What kind of college teaches about load balancers?

If they’re not teaching about the most basic principles of building scalable systems, what is the point of doing a degree, or hiring people who have done a degree?

In the 90s my uncle was disgusted with me because I couldn’t fix his TV — didn’t I have an EE degree?

Re: Interns with toasters: how I taught people about load balancers

#34

Post by jakobegger » Sun, Apr 22, 2018, 11:57 AM UTC

amingilani wrote: post #2

Who is Rachel and how do her short and simple stories always hit the front-page? They're interesting but I always think they're a little _too_ simple. I mean this entire thing can be summed up with: 500s (and other errors) are returned faster than processed requests. Load-balancers will find a misbehaving server's queue empty more often and give it all the requests

Everything seems trivial once you understand it. The author seems to be good at making people understand, so that it seems trivial afterwards.

Knowing nothing about load balancers, had I read your summary before Rachels article, I would probably just glossed over it, but I would not have understood its profound importance for designing load balancers.

Rachel's story kept me engaged and explained the problem in a way that I will surely remember a lot longer.

Teaching is not about reciting facts -- that's what reference manuals are for. Teaching is for engaging students, making them interested, giving background on a topic, telling you why something matters, making it easy to understand, and making sure the student remembers the most important bits.

It's something I have been struggling a lot with junior devs and interns. I show them what they are doing wrong, how to do it correctly, and the next day they make the same mistake again. If people were perfect logical machines, just telling them facts might be sufficient. Unfortunately people are people, and it's really really hard to teach them something effectively.

Re: Interns with toasters: how I taught people about load balancers

#35

Post by matt4077 » Sun, Apr 22, 2018, 11:58 AM UTC

grosjona wrote: post #22

That's why I like random load balancing. If each machine is powerful enough and can handle a few thousand users then the distribution averages out. Smart load balancers are only really necessary if you have inefficient servers that can't handle more than 100 connections per second and they're difficult to get right. If you toss a coin 10 times, you're much more likely to get >=80% heads than if you were to toss that…

Random balancing only works with homogenous servers.

Re: Interns with toasters: how I taught people about load balancers

#36

Post by matt4077 » Sun, Apr 22, 2018, 12:04 PM UTC

BillinghamJ wrote: post #12

rajacombinator wrote: post #8

Earlier quoted context omitted.

What kind of college teaches about load balancers?

If they’re not teaching about the most basic principles of building scalable systems, what is the point of doing a degree, or hiring people who have done a degree?

The S in CS is science. Load balancers have nothing to do with the science of computation. It’s just like no self-respecting university will teach PowerPoint or “fixing Windows” in their CS curriculum.

Re: Interns with toasters: how I taught people about load balancers

#37

Post by jakobegger » Sun, Apr 22, 2018, 12:05 PM UTC

grosjona wrote: post #22

That's why I like random load balancing. If each machine is powerful enough and can handle a few thousand users then the distribution averages out. Smart load balancers are only really necessary if you have inefficient servers that can't handle more than 100 connections per second and they're difficult to get right. If you toss a coin 10 times, you're much more likely to get >=80% heads than if you were to toss that…

That model works if you have a large number of short lived requests.

On the other hand, if you have a small number of long running requests that you want to distribute over a number of servers, then your load balancer needs to track which servers are busy, or you'll spend a lot of time waiting for the randomly assigned server to finish, while others are idle.

Re: Interns with toasters: how I taught people about load balancers

#38

Post by darshitpp » Sun, Apr 22, 2018, 12:13 PM UTC

> This is what happened when one bad web server decided it was going to fail all of its requests, and would do so while incurring the absolute minimum amount of load on itself.

Good ELI5 explanation, but it doesn't really explain why the webserver failed the requests as it did. Or maybe I'm missing something?

Re: Interns with toasters: how I taught people about load balancers

#39

Post by snovv_crash » Sun, Apr 22, 2018, 12:17 PM UTC

This reminds me strongly of another system of distribution, which suffers from the same effect: differential gearboxes in cars.

Since the torque to all the wheels is equal, if one wheel slips it very quickly takes all of the power of the engine, since power = rotational velocity * torque. Only if the speeds are similar is the allocation of power to the wheels similar.

In cars, the solution is to make sure that the power is distributed fairly evenly even if a wheel loses traction. This used to be done with thick grease inside the differential gearbox to limit the difference in speeds - more recently complex gearboxes and traction control achieve the same thing, but through constantly monitoring the speeds of each wheel and braking wheels that are spinning out of control.

It's interesting how such disparate distribution systems seem to have such similar failure modes. I wonder if the two sides could learn something from each other here.

Re: Interns with toasters: how I taught people about load balancers

#40

Post by snovv_crash » Sun, Apr 22, 2018, 12:21 PM UTC

amingilani wrote: post #2

Who is Rachel and how do her short and simple stories always hit the front-page? They're interesting but I always think they're a little _too_ simple. I mean this entire thing can be summed up with: 500s (and other errors) are returned faster than processed requests. Load-balancers will find a misbehaving server's queue empty more often and give it all the requests

They are short and simple, but looking from her blog it seems she also writes one every single day.

Given that the subject matter strongly overlaps with what HN is about and the quality is high, I'm not surprised it gets HN attention fairly frequently.