What I don't understand is since it's the same service can't we just run a performance test, determine the ballpark of the needed resources and just have them capped like in k8s? why would some servers be more powerful than others in the first place?
Load Balancing
101–110 of 243 posts
Re: Load Balancing
#102Fantastic post; useful and entertaining. 10/10, bookmarked, will share.
Context: perf-related webdev since 1998.
Re: Load Balancing
#103Well written and animated!!
Anyone knows what technology was used for the animation?
Re: Load Balancing
#104What I don't understand is since it's the same service can't we just run a performance test, determine the ballpark of the needed resources and just have them capped like in k8s? why would some servers be more powerful than others in the first place?
Some requests may require utilizing different services than others.
Re: Load Balancing
#105Earlier quoted context omitted.
Nice to see someone mention direct server return or as BigIP called it nPath routing. This was an effective scaling method for handling small request that returned large payloads (audio and video files). I don't know how well known this configure is or whether it is still viable in an all TLS world.
It doesn't seem particularly well known. It works fine with TLS, but the origin servers need to do the TLS termination (IMHO, this is better for security than having your load balancers do it, but it does mean you have to work harder on key distribution). On a non-DSR load balancer, doing TLS termination on the origins means the load balancer has less application data to work with (request path, response status code,…
Should you wish to give DSR load balancing a go without having to invest in hardware/licenses you could try https://github.com/davidcoles/vc5
Put that in front of some HAProxy servers to do TLS termination and farm out requests to another layer of NGINX/uWSGI boxes and Robert is your cousins father.
Re: Load Balancing
#106What I don't understand is since it's the same service can't we just run a performance test, determine the ballpark of the needed resources and just have them capped like in k8s? why would some servers be more powerful than others in the first place?
Some requests may require utilizing different services than others.
Yeah, this for sure. Traffic mix changes over the life of an application. In k8s performance isolation certainly isn't a solved problem, and noisy neighbours can affect the "power" of a server.
Any change to the application at all could upset the delicate balance of its performance. You would need to be doing this sort of profiling on every change, using up to date and accurate production traffic.
It ends up being much easier to put some smarts in your load balancer.
Re: Load Balancing
#107Well written and animated!! Anyone knows what technology was used for the animation?
I used https://pixijs.com/. This was my first project with it and it was really nice to use. I fell in to a few traps here and there, but generally it wasn't too bad to get the results I wanted with it. :)
Re: Load Balancing
#108What I don't understand is since it's the same service can't we just run a performance test, determine the ballpark of the needed resources and just have them capped like in k8s? why would some servers be more powerful than others in the first place?
‘Noisy neighbor’ VMs and different underlying hardware specs on the ‘same’ instance configuration are two examples that come to mind.
Re: Load Balancing
#109Interactive diagrams and visuals are worth so much with these topics. I just created documentation at ngrok (full disclosure, I work there!) for how to do load balancing with their edges product for this the other day! Now I have something to refer people to when describing these.
https://ngrok.com/docs/guides/how-to-round-robin-load-balanc...
Re: Load Balancing
#110iirc there was a famous case where an outage occurred at a company using some least connection or least resource algorithm in the load balancers. The load balancer queue'd up requests and when a new server was brought online it immediately dos'd it out of existence. Then another server was brought online and the same thing occurred and another and another. They eventually switched to round-robin to give the new servers coming online a fighting chance to survive. Eventually, enough were online to process the queue.
the take away was complex load balancing can make problems worse when bad things happen unexpectedly.