The march away from companies directly managing Kubernetes to Kubernetes being the layer that every future abstraction will be built on continues.
Autoscale Kubernetes workloads on any cloud using any event
31–40 of 46 posts
Re: Autoscale Kubernetes workloads on any cloud using any event
#32Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
Anyone who doesn't have the same load 24 hours a day?
Ed-tech is a big one where you may have extremely low traffic on weekends/summer/holidays/breaks.
Re: Autoscale Kubernetes workloads on any cloud using any event
#33Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
I had a simple one on a Kubernetes cluster in AWS.
What happened is we'd have a queue processor that normally needed a couple of pods to handle events. Except that once a day another process would drop in 5 million requests into the queue.
So I just had a simple keda autoscaler based on the length of the queue. One pod for every 10,000 items in the queue with a minimum of 2 pods and a maximum of 50 pods.
It would scale up after the big queue dumps, chew threw the backlog and then scale back down again.
Re: Autoscale Kubernetes workloads on any cloud using any event
#34Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
We scale our cicd.
~8-~24 Mo-fr 10-350vms
Re: Autoscale Kubernetes workloads on any cloud using any event
#35Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
We work with customers with various workloads and some ways we help them scale:
1. Have a baseline amount of resources and use keda to scale up using spot instances. 2. For video call bots scale up during the calls and then scale down once the call is completed. Some of our customers can scale up to hundreds of machines during the day then scale down to a couple during the end of the business day. 3. Scale up for cronjobs that require a lot of resources but where the web traffic can use significantly less resources.
Re: Autoscale Kubernetes workloads on any cloud using any event
#36Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
This is the cloud so we are talking about paying for servers running/provisioned at any given time.
If you’re cronjob heavy, or if you have consistent cycles of traffic patterns (say it follows the working day), you can save a LOT of money by not running idle servers all night or on the weekends or whatnot. This was like a 40k per month delta at my last gig.
We could also talk about optimizing the costs of development and staging and sales demo workloads that don’t need to run 24/7 or even 8/5 as well.
Re: Autoscale Kubernetes workloads on any cloud using any event
#37Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
I had a simple one on a Kubernetes cluster in AWS. What happened is we'd have a queue processor that normally needed a couple of pods to handle events. Except that once a day another process would drop in 5 million requests into the queue. So I just had a simple keda autoscaler based on the length of the queue. One pod for every 10,000 items in the queue with a minimum of 2 pods and a maximum of 50 pods. It would sca…
Keda was really good. I just wish that they had actually developed an http scaler. Having to use Knative scaling as an alternative is a pain.
Re: Autoscale Kubernetes workloads on any cloud using any event
#38Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
auto scaling could also mean scaling to 0. If you are running GPU workloads in k8's, you would typically setup a node pool with gpus that can scale to 0 after the job runs.
Re: Autoscale Kubernetes workloads on any cloud using any event
#39Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
I gave a talk on this[0], but I've had some moderate success doing autoscaling based on region for AAA always-online games. That said, you could conceivably live at a higher abstraction. Take dev environments for example. Ideally the team working on infra problems does not need to care how many versions of a backend are operating on the dev environment. The only thing infra needs to take into account the requested re…
> Ideally the team working on infra problems does not need to care how many versions of a backend are operating on the dev environment.
> The only thing infra needs to take into account the requested resources.
> Perverse incentives on wasting resources aside
(I do infra.) That's like, 95% of the problem. AFAICT, most devs have absolutely no idea how powerful a computer is.
My last change was to resize a 400 core, 800 GiB set of compute into a 100 core, 150 GiB set. It was just ludicrously over-provisioned, because the dev teams isn't incentivized to care at all. (…sadly, I'm not allowed to go out an hire a dev now, even though I literally just saved that amount of money in cloud costs…)
(It's still over-provisioned, but that was the easy "we can lop this compute off and I promise you you won't notice.)
The economics/incentives at play are the hard part. Getting management to not look at infra for "ehrmagerd the cloud bill" but instead devote dev time into getting them to dig into "why is this app, which ostensibly just shuffles JSON about the landscape, using 8 cores and all the RAM?" is … tough. And not what I signed up for in SWE, damn it.
The other way is equally bad: devs find they've run out of resources? Knee-jerk is "resize compute upwards" not some introspection of "wait, what is a reasonable amount of CPU use for JSON shuffling?"
Usage graphing is the other tool that really puts some devs' work in a rather bad light: resource requests of like 20 CPU, but the usage graph says "0.02 CPU". So … at least the code's not inefficient, but the requested resources are wasted.
Re: Autoscale Kubernetes workloads on any cloud using any event
#40Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.
I have the opposite question: who is on the cloud and has such consistent workloads they never need to scale up or down? I'm sure those users exist but they must be the minority, right?
Well … I don't have a consistent workload (our load is highly diurnal, enough so that you can spot lunch) but we just mostly don't deal with the complexity of it. For many things, we just alloc 3 VMs, across 3 zones — we really don't need more?¹ — and so the next scaling step is to 0.
¹I think the thing here is that for most of the jobs I've worked … we're really not doing "big" things. The complexity is all business logic or how the product does what it does, not scale.
(We do provision some CI compute on-demand, so that scales with load, so it's not all fixed.)
I think the last "ooh, fun compute!" thing I did was like almost 10 years ago now where we had a huge job that needed to run. But it was sort of the opposite of the stuff in this thread: since it wasn't on-demand, we could run it whenever. That ended up being at night on spot-priced VMs, when they were cheap.