TLDR; if app slow, give more resources
Writing better code is of course out of the question.
[dead]
We should get rid of average CPU utilization
11–20 of 25 posts
Re: We should get rid of average CPU utilization
#12Re: We should get rid of average CPU utilization
#13Let’s measure temperature :)
Re: We should get rid of average CPU utilization
#14A more general metric that is useful to watch for is pressure stall information for CPU, IO and Memory.
https://docs.kernel.org/accounting/psi.html
I made a Prometheus exporter for it:
Re: We should get rid of average CPU utilization
#15Re: We should get rid of average CPU utilization
#16No, not at all. Why get rid of a low-level statistical measure? It's not even quite clear what the article argues against. htop doesn't even show you "average CPU utilization", it provides a sample of the current CPU utilization.
To me the problem appears to be that they try to do some hard realtime computing with strict time guarantees, but are so far up the stack (golang library, golang scheduler, docker, kubernetes, virtualization, etc.), that they don't realize that this stack can't guarantee you realtime computing. CPU utilization is a very low-level measure and, in this stack, is only indirectly related to the observed timeouts.
Re: We should get rid of average CPU utilization
#17I've learned the hard way that CPU resource limits in K8S are a bad idea, as can be seen in this post. Just use CPU requests without limits so the scheduler has an estimate of your applications CPU requirements, but it can burst to use more CPU when it's available.
With memory of course you should set a limit and from experience it should be the same as your memory requests.
Re: We should get rid of average CPU utilization
#18Re: We should get rid of average CPU utilization
#19A more general metric that is useful to watch for is pressure stall information for CPU, IO and Memory. https://docs.kernel.org/accounting/psi.html I made a Prometheus exporter for it: https://github.com/arianvp/cgroup-exporter
Yes!
Re: We should get rid of average CPU utilization
#20I've learned the hard way that CPU resource limits in K8S are a bad idea, as can be seen in this post. Just use CPU requests without limits so the scheduler has an estimate of your applications CPU requirements, but it can burst to use more CPU when it's available. With memory of course you should set a limit and from experience it should be the same as your memory requests.
There is also the concern that a single pod shouldn’t be able to take down an entire node. So there needs to be some safety levels. But then also not. I find this is a really complex issue which is not widely known (only in Kubernetes bubble)