Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Load Balancing

samwho.dev

Post reply on HN
Print view View original

11–20 of 243 posts

Re: Load Balancing

#11

Post by choult »

jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

That's called sharding; this can be achieved with any load balancer worth its salt.

Re: Load Balancing

#12

Post by lucideer »

choult wrote: post #11
jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

That's called sharding; this can be achieved with any load balancer worth its salt.

This is called sharding, but the question is more about the assignment algorithm(s) supported by the sharding algorithms, which are not always flexible enough to support the gp's suggestion.

Re: Load Balancing

#13

Post by mkl95 »

jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

Sounds like sticky sessions

Re: Load Balancing

#14

Post by yabones »

jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

That's very common with stateful applications. Lots of people use HAProxy or other application-aware LB's to keep the sessions "sticky" to a single app server.

https://www.haproxy.com/blog/enable-sticky-sessions-in-hapro... https://www.haproxy.com/blog/load-balancing-affinity-persist...

Re: Load Balancing

#15

Post by bbatchelder »

jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

Did this 20 years ago by having the name of the server as part of the user's profile.

User's 1 through 50 (light users) log in and their profile says they go to app-1.myapp.com. User's 51 through 60 (heavy users) log in and their profile says they go to app-2.myapp.com.

A specific user may pay extra to have a non-shared environment, and this supports that as well.

Re: Load Balancing

#16

Post by bbatchelder »

The playground simulation is pretty cool.

One thing I found interesting, is it you go with PEWMA and create a scenario where the cluster is stressed, and then add 1 server, it pummels the shit out of the new server and you have a brief surge in failed requests.

Not sure if that is a real world issue, or just with the simulation...

Re: Load Balancing

#17

Post by Mavvie »

throwaway71271 wrote: post #8

we are so stuck with this push request load balancing its crazy, if we just switch to pull instead of push things get much smoother, and resources get better utilized you cant reliably guess if the instance where you will push your request actually has capacity to handle it, even using ML to guess it will still have thrashing properties but if you just let instances pull work, things work out for themselves sadly, th…

So what do you think the disadvantage is of a pull approach? Presumably it's not just better or else tools would use it?

Re: Load Balancing

#18

Post by jqpabc123 »

bbatchelder wrote: post #15
jqpabc123 wrote: post #10

I have a simple alternative method in mind for SAAS (software as a service) apps. Manual/statistical load balancing --- assign users to a specific server based on their login credentials. A statistical model of server utilization can be maintained and users assigned or re-assigned as needed. Latency can be reduced to zero by simply forwarding the connection to the proper server once the login is complete. The obvious…

Did this 20 years ago by having the name of the server as part of the user's profile. User's 1 through 50 (light users) log in and their profile says they go to app-1.myapp.com. User's 51 through 60 (heavy users) log in and their profile says they go to app-2.myapp.com. A specific user may pay extra to have a non-shared environment, and this supports that as well.

Did this 20 years ago by having the name of the server as part of the user's profile.

I had a simple lookup table in mind so it can be easily changed and adjusted as required without affecting the user's profile.

Re: Load Balancing

#19

Post by gbuk2013 »

throwaway71271 wrote: post #8

we are so stuck with this push request load balancing its crazy, if we just switch to pull instead of push things get much smoother, and resources get better utilized you cant reliably guess if the instance where you will push your request actually has capacity to handle it, even using ML to guess it will still have thrashing properties but if you just let instances pull work, things work out for themselves sadly, th…

Moving to async model adds new set of operational challenges as well as some interesting failure scenarios. (Edit) Also, in practice you would need at least one more system to enqueue request into the broker, as the latter would typically not be exposed to the outside world.

Request/response on the other hand is much simpler to configure and operate.

Re: Load Balancing

#20

Post by deepfriedrice »

mnutt wrote: post #7

The simulations are really cool. I'm trying to think of a real-world situation where you'd have differently-sized backends though. I guess it might help if your server runs hourly cron jobs that take away system resources from requests.

Had the same thought. With AWS et al. it's just too easy to use the same instances and avoid that variable.
Post reply on HN