Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interns with toasters: how I taught people about load balancers

rachelbythebay.com

Post reply on HN
Print view View original

81–90 of 124 posts

Re: Interns with toasters: how I taught people about load balancers

#81

Post by indigodaddy »

The rachelbythebay.com new_blog_post auction:

1) Navigate to rachelbythebay.com and refresh until new blog post appears 2) Submit new rachelbythebay.com new_blog_post to HN 3) Crosseth fingers and hope that your HN post is the one that gains HN traction, out of the five or ten others that have also just posted the rachelbythebay.com new_blog_post. 4) Profit, as yours is the one! Receive much HN points!

I'm ready for my downvotes now, but this is a fairly good illustration as to how popular almost any rachel post is on HN. Her articles are actually quite good and usually engaging to the reader, and have that certain knack/quality that endears itself to the HN readership.

Re: Interns with toasters: how I taught people about load balancers

#82

Post by trjordan »

mabbo wrote: post #72
vasco wrote: post #4

And that's why you always set health checks on servers behind a load balancer that should take bad ones out as soon as possible. You then get another interesting problem, which is if the server is only spitting errors when serving requests, it'll then be marked healthy again and forever toggle between healthy and unhealthy. But that you can also solve.

The mistake I've seen here is having the "health check" do little or nothing. In many web services, I've seen a /ping or /health api that the load balancer calls to ask "are you healthy?". But people get rushed or lazy or requirements change, and they wind up with that api just doing "return true". Now you've got a host that can't access a database or can't do much of anything- but it can return true! Health check AP…

Outlier detection also fixes this, without having to implement health checks that replicate the whole functionality (complexity) of the rest of the service.

10 500s in a row? Go to the timeout chair until you get better.

Re: Interns with toasters: how I taught people about load balancers

#83

Post by vasco »

randerson wrote: post #78
s_kilk wrote: post #74

Earlier quoted context omitted.

At work we have two endpoints on every service, `/status` and `/health-check`. The former is basically a "return true" endpoint, which can tell you if the service is alive and reachable. The latter will usually do something like "select 1;" from any attached databases and only succeed if everything is OK.

And the former is the one you want your Load Balancers to be checking. With a deep health check, even a brief database outage will cause every web server to be taken out of rotation, and then you're completely down for at least as many health check intervals as it takes for the LB to consider a host healthy again. Same goes for any other shared resource that is likely to affect all web servers if it becomes unavailab…

Presumably your service would not be in a very useful state anyway if the main data store it needs to function is out, so which kind of health-check you use will depend on the failure mode you want to expose to your users.

Re: Interns with toasters: how I taught people about load balancers

#84

Post by grosjona »

jakobegger wrote: post #37
grosjona wrote: post #22

That's why I like random load balancing. If each machine is powerful enough and can handle a few thousand users then the distribution averages out. Smart load balancers are only really necessary if you have inefficient servers that can't handle more than 100 connections per second and they're difficult to get right. If you toss a coin 10 times, you're much more likely to get >=80% heads than if you were to toss that…

That model works if you have a large number of short lived requests. On the other hand, if you have a small number of long running requests that you want to distribute over a number of servers, then your load balancer needs to track which servers are busy, or you'll spend a lot of time waiting for the randomly assigned server to finish, while others are idle.

If each server can handle thousands of operations at a time, the duration of each operation shouldn't matter because long operations should also be evenly spread across servers... Also due to probability.

When you're dealing with big numbers across relatively few servers, randomness averages out everything.

Re: Interns with toasters: how I taught people about load balancers

#85

Post by kd5bjo »

grosjona wrote: post #84
jakobegger wrote: post #37

Earlier quoted context omitted.

That model works if you have a large number of short lived requests. On the other hand, if you have a small number of long running requests that you want to distribute over a number of servers, then your load balancer needs to track which servers are busy, or you'll spend a lot of time waiting for the randomly assigned server to finish, while others are idle.

If each server can handle thousands of operations at a time, the duration of each operation shouldn't matter because long operations should also be evenly spread across servers... Also due to probability. When you're dealing with big numbers across relatively few servers, randomness averages out everything.

It depends on how long your long operations are. Back when I was working on Justin.tv, we had to write a custom load balancer because we had to maintain a sizable number of TCP connections that lasted for hours or days without disconnecting.

Re: Interns with toasters: how I taught people about load balancers

#86

Post by gaius »

randerson wrote: post #78
s_kilk wrote: post #74

Earlier quoted context omitted.

At work we have two endpoints on every service, `/status` and `/health-check`. The former is basically a "return true" endpoint, which can tell you if the service is alive and reachable. The latter will usually do something like "select 1;" from any attached databases and only succeed if everything is OK.

And the former is the one you want your Load Balancers to be checking. With a deep health check, even a brief database outage will cause every web server to be taken out of rotation, and then you're completely down for at least as many health check intervals as it takes for the LB to consider a host healthy again. Same goes for any other shared resource that is likely to affect all web servers if it becomes unavailab…

Databases are 10-100x as reliable as the application tier in my experience

Re: Interns with toasters: how I taught people about load balancers

#87

Post by gaius »

pulkitsh1234 wrote: post #23

Maybe some people dismissed her problem as 'impossible' because she didn't inform what kind of load balancing technique was the load balancer using? I suspect what happened is that they didn't understand the problem, and so resorted to ineffective means to steer the attention away from their own inadequacies. ^ This statement is kinda harsh.

Every post on that blog is about how the author is incredibly clever and surrounded by dolts.

Re: Interns with toasters: how I taught people about load balancers

#88

Post by antoncohen »

snovv_crash wrote: post #39

This reminds me strongly of another system of distribution, which suffers from the same effect: differential gearboxes in cars. Since the torque to all the wheels is equal, if one wheel slips it very quickly takes all of the power of the engine, since power = rotational velocity * torque. Only if the speeds are similar is the allocation of power to the wheels similar. In cars, the solution is to make sure that the po…

Chevy made a brilliant video in 1937 explaining how differentials work:

https://www.youtube.com/watch?v=yYAw79386WI

Re: Interns with toasters: how I taught people about load balancers

#89

Post by walshemj »

mrgordon wrote: post #24
BillinghamJ wrote: post #12

Earlier quoted context omitted.

If they’re not teaching about the most basic principles of building scalable systems, what is the point of doing a degree, or hiring people who have done a degree?

Several of the top CS programs specifically do not teach details like configuring a load balancer as with a proper background in computer science it is assumed that you can figure out the latest programming languages and the install instructions for popular software packages. It would thus be a waste of a very expensive degree as you could learn about configuring LAMP and a load balancer at a trade school instead of…

That is what you get the technicians to do

Re: Interns with toasters: how I taught people about load balancers

#90

Post by rollcat »

hluska wrote: post #57
rollcat wrote: post #48

Earlier quoted context omitted.

Then you design your system to always respond as quickly as possible. HTTP even has a status code for that: 202 Accepted. You put the job in a queue and return some ID or cookie to the client, possibly also an ETA. Client can then poll for results (with bounded exponential back-off), again polling is just a quick key look-up; alternatively, if you have some means to push results to clients (like via WebSockets), use…

I've spent enough time building systems and doing ops that this complexity genuinely scares me. Have you used this pattern in highly scalable production apps? What's it like to debug when everything goes to hell?

So first, this is not my idea, but something that people much smarter than me came up with.

This is the pattern that AppEngine enforces. Apps that regularly take more than 1s to respond are penalized, and there's a 60s hard deadline on all requests. Things like a task queue or push channels are built-in to the platform, with a high-level API exposed, so you just focus on writing the application / business logic.

Celery tries to basically do the same for you, w/o the proprietary Google SDK's.

> What's it like to debug when everything goes to hell?

From my experience so far... You will find bugs in your application code way more often than in a battle-tested task queuing system, much like you're quite unlikely to find a bug in nginx or Python.

Fixing things is MUCH easier than in a request / response model. I know it's not a "web scale" example, but imagine an installation involving camera rigs, some networked hardware, a transcoding server, etc. Dude walks into a photo booth, types in his email/phone number on a tablet, etc. When I see a failure in a component of the pipeline, I can hack together and deploy a patch in seconds / minutes (from alert to running the fix in production) and just tell the system to retry a failed job. Dude gets his silly photo via email/sms within two minutes instead of one.

Picture a similar situation in a web application that processes user-uploaded media. You write your batch processing logic to bail early on any sign of error, and when you get the chance to fix some bug that e.g. affected 3% of your users, again - you resubmit the failed jobs and users just see the result with a delay, instead of having to re-upload.

Post reply on HN