Autoscale Kubernetes workloads on any cloud using any event

> If you have the hardware resources, why not just scale up from the beginning on?

For most workloads it's wasteful to have max capacity provisioned at all times if you can instead provision on-demand.

This is true in general. For example, electricity supply is a mix of baseload power (cheap but only if left running constantly) and peaking (expensive but easy to turn on and off). It wouldn't be economical to have baseload capacity equal to maximum demand. Instead it is aimed at minimum demand and other sources make up the difference depending on demand.

For people on the cloud, you can buy new computers and sell them back in a period of time measured in hours, so that's how people are using these systems. It doesn't make a ton of economic sense to me in general, because your periods of high demand are going to be the same periods for everyone else in that datacenter (you want to serve close to your users to minimize latency, and most of the users in a geographical region go to work and sleep at the same times). That said, it's not priced like that. Having burstable core instances "always on" can end up being more expensive than buying guaranteed capacity instances for a short period of time.

I never auto-scale interactive workloads, but it's good for batch work.

Other people have different feelings. Consider the case where you release software multiple times a day, but it has a memory leak. You don't notice this memory leak because you're restarting the application so often. But the Winter Code Freeze shows up, and your app starts running out of memory and dying, paging you every day during your time off. If you had horizontal autoscaling, you would just increase the amount of memory that your application has until you come back and fix it. Sloppy? Sure. But maybe easier to buy some RAM for a couple weeks and not disrupt people's vacation. (The purist would argue their vacation was ruined the day they checked in the memory leak.) This gets all the more fun when the team writing the code and the team responsible for the error rate in production are different teams in different time zones. I don't think that's a healthy way to structure your teams, but literally everyone else on earth disagrees with me, so... that's why there's a product that you can sell to the infrastructure team instead of telling the dev team "wake up and call free() on memory you're not using anymore".

Our customer workloads are bursty, so being able to scale down to 0 (or close to it) saves us a lot of CPU and memory that would otherwise do nothing for most of the day.

We’re using KEDA and ScaledJob to scale tomographic reconstructions in the cloud. When a CT scanner has finished uploading a scan, we let a ScaledJob create a Job to process the data. A scan is maybe 8 hours and during that time we don’t need any compute resources. But when it’s done we need both lots of CPU and GPU power to process GBs and TBs of data rapidly to show previews to the user.

Also, when a user triggers new previews we scale up nodes to process that data. The problem there though is the scale up time of the node pool which is a few minutes for a GPU node on Azure.

We payed to have a GPU running all the time before but that got too expensive.

As a side note, would I do it again I probably wouldn’t build a data pipeline on top of KEDA ScaledJobs and possibly not use Kubernetes at all.

No it’s 2025, but definitely going to happen.