Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#1

Post by mmanciop » Wed, Jan 08, 2025, 11:59 AM UTC

Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool
dash0.com

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#2

Post by mmanciop » Wed, Jan 08, 2025, 11:59 AM UTC

I looked into how to wire the Horizontal Pod Autoscaler for Kubernetes to Dash0, the OpenTelemetry-native observability tool I am working on, and it turned out to be surprising simple.

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#3

Post by bushbaba » Sat, Jan 11, 2025, 3:10 PM UTC

While useful, this is a more fragile autoscaling solution that HPA out of the box. It requires your otel metric to Prometheus pipeline to be up AND your Prometheus server to be up.

Compare that to the vanilla HPA which is entirely cluster local, and “simple”.

Generally the simple approach is the most stable approach.

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#4

Post by pm90 » Sat, Jan 11, 2025, 3:44 PM UTC

keda also works for this and supports many backends

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#5

Post by dilyevsky » Sat, Jan 11, 2025, 7:23 PM UTC

bushbaba wrote: post #3

While useful, this is a more fragile autoscaling solution that HPA out of the box. It requires your otel metric to Prometheus pipeline to be up AND your Prometheus server to be up. Compare that to the vanilla HPA which is entirely cluster local, and “simple”. Generally the simple approach is the most stable approach.

HPA still requires metrics-server to be up which it’s using to obtain current pod resource usage

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#6

Post by mmanciop » Sat, Jan 11, 2025, 7:26 PM UTC

bushbaba wrote: post #3

While useful, this is a more fragile autoscaling solution that HPA out of the box. It requires your otel metric to Prometheus pipeline to be up AND your Prometheus server to be up. Compare that to the vanilla HPA which is entirely cluster local, and “simple”. Generally the simple approach is the most stable approach.

True, but that approach is also limited in what it can autoscale on, namely Utilization metrics. They are not always good data to make scaling decisions on (covered in the article).

Re: Kubernetes horizontal pod autoscaling powered by an OpenTelemetry-native tool

#7

Post by mmanciop » Sat, Jan 11, 2025, 7:39 PM UTC

pm90 wrote: post #4

keda also works for this and supports many backends

Yes, it probably would be able to do the same logic as in the blogpost with the Prometheus scaler: https://keda.sh/docs/2.16/scalers/prometheus/

I might give it a go in a follow-up :-)