Live data from Hacker News

Autoscale Kubernetes workloads on any cloud using any event

kedify.io

41–46 of 46 posts

Re: Autoscale Kubernetes workloads on any cloud using any event

#41

Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.

I'm not sure if this answers your question but the last 2 companies I worked at (~7 years) both had very clear traffic spikes 9a-5p US east coast hours on weekdays. My current place actually sees more than 20-30% drop sunday nights compared to monday morning, and it's constantly going up because we have a lot of American enterprise customers. Maybe I misunderstood your question but is there a case where you can keep…

> is there a case where you can keep your entire capacity running for free? I'd assume you pay AWS/other cloud or your electricity provider.

Colo providers charge by the [rack with given network port size and power delivery], so unless you literally host on premises which almost nobody does even when they talk about on prem, once you get outside of a cloud environment it is rare for it to pay to shut down servers unless they'll be down for a long time. Maybe there'd be a business there for colo providers to offer pricing that incentivises powering down machines (almost all modern servers have IPMI, and so as long as you provide the trickle - relatively speaking - of power for the IPMI board you an power the servers down/up over the network on demand), but it's not the norm.

The problem with these traffic spikes you mention is that "everybody" has them, and the overlap is significant, and so they're priced in because the cloud providers needs capacity to handle the worst overlap in spikes, plus margin. 20%-30% drop is way too low to cover the cost gap between even managed servers with a huge capacity margin and most cloud providers. I've worked for a lot of different companies where we've forecast our capacity requirements, and the graphs look almost identical. Sometimes shifted n hours to account for different in timezones, but for a lot of companies the graph is near identical globally because of similar distributions of userbase.

(If you do think you can do scaling up/down for daily spikes cost effectively, you can typically do it even more cost effectively by putting your base load in a colo'ed environment, and scale into a cloud environment if you hit large enough spikes; the irony is that in environments where I've done that, we've ended up cutting the cost of the colo'ed environment by cutting closer to the margin and end up almost never end up scaling into the cloud, but it gives peace of mind, and so being prepared to use cloud has made actually using cloud services even less cost attractive).

In practice, most places - there are exceptions that makes good use of it - just set up autoscaling so they don't need to pay attention to creeping resource use. Which is rarely a good use of it.

There are good uses for autoscaling, but it's very rare for day/night or weekend/weekday cycles to be significant enough that it isn't still cheaper to buy enough capacity to take all or most of the spikes (but having the ability to scale into a cloud service might mean you only buy just enough for the "usual" weekday cycles, or even shave a little bit of the top, instead of buying enough for unexpected surges on top).

Re: Autoscale Kubernetes workloads on any cloud using any event

#42

Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.

This is the cloud so we are talking about paying for servers running/provisioned at any given time. If you’re cronjob heavy, or if you have consistent cycles of traffic patterns (say it follows the working day), you can save a LOT of money by not running idle servers all night or on the weekends or whatnot. This was like a 40k per month delta at my last gig. We could also talk about optimizing the costs of developmen…

I think this is part of the problem. We're talking about the cloud because "everyone" think they a) need autoscaling (some do, many don't, and even fewer would need it if they weren't paying inflated cloud costs), b) never consider that they can keep base load on servers provisioned one way and auto-scale additional capacity only, and actually considering that tends to change the economics dramatically.

I've set up multiple hybrid setups over the years, and what I've consistently found is that we can provision 2x-3x (more if egress is high) the amount of server capacity for the same price with managed hosting providers or in colo'ed environments than with cloud providers. That's fully loaded cost including rates for contracts for devops etc..

Very few people need to auto-scale up more than that. But since most people still want orchestration, it tends to cost little to set up their system so that if they have a spike, they can scale up extra capacity in a cloud. And in doing so, they can cut the amount of hardware they provide for the base load to whatever is cheapest.

The first times I did this, I was fully convinced going in that this would mean we'd set the base load around the lowest utilization over a typical 24/7 cycle, and spin up some cloud instances during daily peaks etc.

In practice, after actually testing what pays for a given scenario, I've yet to see that (scaling up/down for a typical 24/7 cycle or 8/5) pay off, though I'm sure it can for some people. .

Managed servers proved in actual, real-life usage scenarios, to be sufficiently cheaper that unless your spikes were very brief and sharp[1], it was cheaper to provision enough base capacity to handle most or all of the normal daily spikes, and what a hybrid setup bought us was the freedom to not overprovision for "what-if" scenarios.

That effect was significant enough that even e.g. SaaS services used almost entirely by office staff within a single time zone often do not save on auto-scaling vs. non-cloud servers scaled for peak use, because the 8-10 hour window of use that creates is far too long - depending on your specific cloud cost and what your cheapest alternative is, it may vary, but I've rarely found it pays to spin up cloud resources for spikes that are on average any longer than 4-6 hours in a day, and that tends to rule out most "normal" cyclical use, especially as you can often adjust "cronjob heavy" parts of your workload to fall outside of the window, for example, to even out the load.

Auto-scaling absolutely pays at far smaller variations in load if your only option is to have all your load in a cloud environment, but even then I see a lot of people resort to auto-scaling before they've even though of cutting the cost of their base load by e.g. ensuring they use reserved instances where it makes sense etc., or negotiating. Often ticking those boxes will have a much larger impact.

By all means ensure your system is built so that it can handle auto-scaling gracefully, though - it will benefit you whether or not you end up making much use of it.

[1] As an example where we got "close", one company I worked had several clients that did large e-mail sends for restaurant chains that often included massive discounts. On the e-mails with highest open rates, you'd then very predictably get traffic spikes from 8:00-8:15, 9:00-9:15, and 10:00-10:15 that were massively, as people checked their e-mail when they got into the office, with the 9:00-9:15 peak being several times higher than their normal daily use. If they had been hosting this themselves, it'd have paid to auto-scale into a cloud env. to handle those spikes, especially as they didn't send such campaigns every day. In our case, most of our other customers had reasonably quiet mornings and so we can could overprovision VM's from our base capacity for them at no extra cost to us. But this was also a rare exception.

Re: Autoscale Kubernetes workloads on any cloud using any event

#44
post #12

kubernetes is dying, isn't it?

Kubernetes is a framework, it'll take a long time to die, it will likely contort itself into fitting whatever paradigm is needed. However, I wonder what you mean? Kubernetes from where I sit has almost complete ubiquity across most companies. Even in places where it's a poor fit.

People starting to put abstractions over it means a) there's a chance people will start asking for a given abstraction rather than Kubernetes, and not care if that abstraction eventually subsumes or replaces Kubernetes, b) at least some people think Kubernetes is enough of a nuisance to deal with to be looking for alternatives.

Whether that means Kubernetes is dying, I'm not so sure. But Kubernetes is extremely complex for a lot of workloads that it's total overkill for, so I'm not surprised people are looking for options.

Re: Autoscale Kubernetes workloads on any cloud using any event

#45

Who needs autoscaling? I mean this as a serious question. Has somebody a real story where autoscaling helped out the company or product? If you have the hardware resources, why not just scale up from the beginning on? If you do not have the resources, you need a lot of money anyways to pay the upscaled rent afterwards.

I worked on a project with an e-commerce company a few months back. They use autoscaling in their Kubernetes cluster to account for high load during peak hours (generally noon until 7PM or so). It would go to 10-ish instances of 2 apps during this peak, and then during the non-peak times it'd drop back down to 2 instanches.

This is pretty significant, since the 2 different apps are relatively large JVM apps, each requiring ~16GiB of memory

Post reply on HN