What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
1–10 of 22 posts
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#2In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort!
It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget.
If you pair the HPA with a decent node autoscaler, THAT in my opinion is the game changer of cloud managed kubernetes over the bare metal deployments that I have done.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#3In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#4In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
> Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans.
It's ill-suited for anyone unless P=NP with a nice solution.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#5In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
I think the boundary makes a lot of sense. Cluster autoscaling only responds to scheduling pressure; if there are pending pods, a new node is added to the cluster so those pods can run. Meanwhile, horizontal pod autoscaling is a totally different system; it adds pods for that service when system-level metrics indicate that it should. Vertical pod autoscaling is again mostly unrelated; if metrics indicate that a certain pod should be bigger, a bigger version is scheduled.
I do see why more integration would be useful, though, including disruption budgets. Mostly for consolidating the incremental cluster autoscaling results onto one node from time to time, without waiting for the workload to naturally disappear or decrease in scale. Also, it would be nice to say "hey if ARM spot nodes are cheaper than AMD64, just reschedule these workloads onto ARM". Basically, it's still the very early days of optimizing cost, latency, and throughput.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#6In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
It's more reliable this way. N number of pods are scheduled on M number of nodes, and if there are multiple sets of pods that each have their own scaling parameters (target utilizations, scaling cooldowns, etc), there is not always a one-to-one mapping with how many nodes are needed.
The cluster autoscaler already has a fairly complex logic just in its own control loop. It uses predicate logic and a simulated scheduler to determine whether a pending pod, based upon node selector, affinity, anti-affinity, taints, tolerations, qos, priority whether expanding a nodegroup would make the pod schedulable.
So it's actually easier (at least for me) to reason out what might happen, with two control loops that work independently in adjacent dimensions than a single one that tries to cover everything. I would not want HPA, VPA, and cluster-autoscaler to be one thing.
I have never used VPA, and in our use-case, we do a different kind of vertical scaling. (Different deployments that target different nodegroups with differently-sized number of cores on the base machine)
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#7Earlier quoted context omitted.
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
I think the boundary makes a lot of sense. Cluster autoscaling only responds to scheduling pressure; if there are pending pods, a new node is added to the cluster so those pods can run. Meanwhile, horizontal pod autoscaling is a totally different system; it adds pods for that service when system-level metrics indicate that it should. Vertical pod autoscaling is again mostly unrelated; if metrics indicate that a certa…
The cluster autoscaler will do pod compaction. It would be nice to specify when to favor more compaction than expansion because you know the traffic is going to fall off after a certain time during the day.
The main thing the integration helps with is reducing the startup time when there is scheduling pressure. If you know your increase in number of pods will always mean an expansion in the nodegroup, you can proactively and optimistically expand the nodegroup.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#8In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
> I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
Go and write it, there are a bajillion open source controllers for Kubernetes that add a ton of value.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#9In my experience, HPA are awesome! Once you defined your sweetspot of buffer pods for quick scaling, they are well worth the effort! It the super simple stuff, scaling down staging on the weekend or even scaling all feature deployments to 0, when you know nobody will be working on it, that will end up saving you big bucks on your cloud budget. If you pair the HPA with a decent node autoscaler, THAT in my opinion is t…
I'm surprised to hear you say that you like having two layers of autoscaling rather than it being some accidental complexity that you just have to put up with because of how the different systems intersect. Having multiple non-orthogonal dimensions of scaling to me always feels like a task ill-suited to humans. I kinda wish HPA, VPA, and Cluster Autoscaler were all just be one thing.
Of course a complete analysis needs something like kubecost running for a couple of weeks to determine where your peaks really are and a couple more for actual fine tuning, but i think its well worth it in the end.
Node autoscaling works best for me with buffer nodes depending on resources and having "one more than you need" is super easy in the cloud.
Dont get me wrong there is still plenty of room for improvenment, but the hard part is defenitly just finding out how much resources your app really needs.
And of course, the application needs to be able to handle scaling to begin with.
Re: What Is Kubernetes HPA and How Can It Help You Save on the Cloud?
#10If the author is here: The illustration in "How does Horizontal Pod Autoscaler work?" section has incorrect before/after CPU utilization % based on the text/logic.