Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
1–10 of 20 posts
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#2I was going through Opencost documentation which this project uses and it looks you need to setup AWS Athena if you want the cloud cost to be displayed for AWS: https://www.opencost.io/docs/configuration/aws#aws-cloud-cos...
Does the Athena does the actual processing/computation of costs ? What is the usual cost for running Athena ?
It also seems strange that I have to put the IAM keys into secrets instead of using IAM role for service account for configuring it.
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#3Is the cost shown only the prices incurred post plugin integration or is there a way to show retroactive costs by comparing k8s object creation dates for example?
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#4So is Headlamp the state of the art in Kubernetes cluster management ever since Mirantis first enshittified Lens and then tucked away its sources?
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#5Is there a way to allocate cost to every pod on a node when node cost is given without break down by resource type and pod resources are not in same ratio as node resources?
Lets say node has 8 CPUs and 32 GB RAM (1:4 ratio). If every pod uses same ratio for its CPU:MEM then math is simple: node cost is split across all pods proportional to their resource allocation.
How to make fair calculation if pod resource ratio is different? In extreme it is still simple - lets say there is a pod with 8 CPU and 2 GB RAM, because no pods can fit into node whole node cost is allocated to that pod.
What if running pod is 6 CPU and 16 GB RAM and another pod with 2 CPU and 16 GB RAM is squeezed in. How to allocate node cost to each? It can't be just node cost / # of pods, because intiutively beefier pods should recive larger share of node cost as they prevent more smaller pods to fit in, but how exactly to calculate it? "weight" of pod on CPU dimenstion is different than on MEM dimension.
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#6So is Headlamp the state of the art in Kubernetes cluster management ever since Mirantis first enshittified Lens and then tucked away its sources?
I didn't try Headlamp, but I moved to AptKube from Lens and been happy since then. It might not be a best in class, but it is snappy and doesn't require any cloud accounts.
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#7So is Headlamp the state of the art in Kubernetes cluster management ever since Mirantis first enshittified Lens and then tucked away its sources?
I really like k9s. It’s plugin model is super easy to work with, if a bit constrained, as well.
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#8I was going through Opencost documentation which this project uses and it looks you need to setup AWS Athena if you want the cloud cost to be displayed for AWS: https://www.opencost.io/docs/configuration/aws#aws-cloud-cos... Does the Athena does the actual processing/computation of costs ? What is the usual cost for running Athena ? It also seems strange that I have to put the IAM keys into secrets instead of using I…
The Cost and Usage Report (CUR) from AWS is just a fine-grained listing of all the resources in your account and their cost. It can be dumped out on different schedules (hourly, daily, monthly) and in different formats (CSV, Parquet).
It is pretty common to configure the CUR files to be dumped into your S3 account and query them via Athena. Athena is billed as $ per TB scanned ($5 last time I looked), so the cost will be based on how often the data is being queried. Downside is that each query can take quite a while to execute depending on data size.
The other common option is to ingest the CUR data into Redshift which gives you better control / options for performance, manipulation, etc. but requires that you set up and manage Redshift.
Hard to tell exactly what the Athena cost here would be as it depends on the number of assets in the account and the frequency in which you are querying the CUR. However, you can issue quite a bit of Athena queries on CUR data for most AWS use cases without incurring too much cost. Unless you have a rapidly changing environment (e.g. hundreds of k of assets turning over daily) or just tons of standing assets, you should be safe to assume hundreds a day at the most? Probably much less for most use cases. This is assuming they are querying once and storing rather than real time querying all the time and normal usage patters, etc.
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#9Is the cost shown only the prices incurred post plugin integration or is there a way to show retroactive costs by comparing k8s object creation dates for example?
I went sniffing around with https://github.com/opencost/opencost/issues?q=is%3Aissue+his...> and https://github.com/opencost/opencost/issues?q=is%3Aissue+ret...> and didn't see anything, so it may be worth creating an issue to put in on their radar. I would also presume you already have the CUR exports for the old times and just want the analysis done using the annotated k8s data?
Re: Kubernetes Cost Management with the New OpenCost Plugin for Headlamp
#10Is there a way to allocate cost to every pod on a node when node cost is given without break down by resource type and pod resources are not in same ratio as node resources? Lets say node has 8 CPUs and 32 GB RAM (1:4 ratio). If every pod uses same ratio for its CPU:MEM then math is simple: node cost is split across all pods proportional to their resource allocation. How to make fair calculation if pod resource ratio…
would like to know if someone's got a more objective approach;
what we currently do is just a maxOf;
take CostPerGB (memory) and CostPerCore (cpu); and costPerPod = max(pod.requests.cpu * CostPerCore, pod.requests.memory * CostPerGB)
at an overall basis, this was ~20% off to actuals when we'd checked this last, so we clearly call out that the costs are "indicative" and not exact.