Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go, Containers, and the Linux Scheduler

riverphillips.dev

Post reply on HN
Print view View original

101–110 of 143 posts

Re: Go, Containers, and the Linux Scheduler

#101

Post by ithkuil »

otterley wrote: post #90

This sort of tuning isn't necessary if you use CPU reservations instead of limits, as you should: https://home.robusta.dev/blog/stop-using-cpu-limits CPU reservations are limits, just implicit ones and declared as guarantees. So let the Go runtime use all the CPUs available, and let the Linux scheduler throttle according to your declared reservations if the CPU is contended for.

I don't set limits because I'm afraid of how a pod is going to affect other pods. I set limits because I don't want to get used to being able to tap on the excess CPU available because that's not guaranteed to be available.

As the node fills up with more and more other pods, it's possible that a pod that was running just fine a moment ago is crawling to a halt.

Limits allow me to simulate the same behavior and plan for it by doing the right capacity planning.

They are not the only way to approach it! But they are the simplest way to so it.

Re: Go, Containers, and the Linux Scheduler

#102

Post by sofixa »

rrdharan wrote: post #51
dekhn wrote: post #16

The common problem I see across many languages is: applications detect machine cores by looking at /proc/cpuinfo. However, in a docker container (or other container technology), that file looks the same as the container host (listing all cores, regardless of how few have been assigned to the container). I wondered for a while if docker could make a fake /proc/cpuinfo that apps could parse that just listed "docker cpu…

Containers are a crappy abstraction and VMware fumbled the bag, is my takeaway from this comment…

> VMware fumbled the bag

Oh they did, they're a modern day IBM.

> Containers are a crappy abstraction

They're one of the best abstractions we have (so far) because they contain only the application and what it needs.

Re: Go, Containers, and the Linux Scheduler

#103

Post by eloisant »

ithkuil wrote: post #101
otterley wrote: post #90

This sort of tuning isn't necessary if you use CPU reservations instead of limits, as you should: https://home.robusta.dev/blog/stop-using-cpu-limits CPU reservations are limits, just implicit ones and declared as guarantees. So let the Go runtime use all the CPUs available, and let the Linux scheduler throttle according to your declared reservations if the CPU is contended for.

I don't set limits because I'm afraid of how a pod is going to affect other pods. I set limits because I don't want to get used to being able to tap on the excess CPU available because that's not guaranteed to be available. As the node fills up with more and more other pods, it's possible that a pod that was running just fine a moment ago is crawling to a halt. Limits allow me to simulate the same behavior and plan f…

That's why you use monitor and alerting, so you notice degraded performances before the pods is crawling to a halt.

You need to do it anyway because a service might progressively need more resources as it's getting more traffic, even if you're not adding any other pod.

Re: Go, Containers, and the Linux Scheduler

#105

Post by FridgeSeal »

Volundr wrote: post #42
dekhn wrote: post #26

Earlier quoted context omitted.

Maybe I should be clearer: Let's say I have a 16 core host and I start a flask container with cpu=0.5 that forks and has a heavy post-fork initializer. flask/gunicorn will fork 16 processes (by reading /proc/cpuinfo and counting cores) all of which will try to share 0.5 cores worth of CPU power (maybe spread over many physical CPUs; I don't really care about that). I can solve this by passing a flag to my application…

It's not clear to me what the max parallelism should actually be on a container with a CPU limit of .5. To my understanding that limits CPU time the container can use within a certain time interval, but doesn't actually limit the parallel processes an application can run. In other words that container with .5 on the CPU limit can indeed use all 16 physical cores of that machine. It'll just burn through it's budget 16…

It won’t burn through the budget faster by having more cores. You’re given a fixed time-slice of the whole CPU (in K8s, caveats below), whether you use all the cores or just one doesn’t particularly matter. On one hand, it would be would nice to be able to limit workloads on K8s to a subset of cores too, on the other, I can only imagine how much catastrophically complex that would make scheduling and optimisation.

Caveats: up to the number of cores exposed to your VM. I also believe the later versions of K8s let you do some degree of workload-core pinning and I don’t yet know how that interacts with core availability .

Re: Go, Containers, and the Linux Scheduler

#106

Post by the8472 »

dilyevsky wrote: post #55
the8472 wrote: post #41

Earlier quoted context omitted.

With shares you're going to experience worse latency if all the containers on the system size their thread pool to the maximum that's available during idle periods and then constantly context-switch due to oversubscription under load. With quotas you can do fixed resource allocation and the runtimes (not Go apparently) can fit themselves into that and not try to service more requests than they can currently execute g…

And how is that different from worse latency due to cpu throttling from your users’ perspective?

Fixed queue, so it'll only take as many as it can process and reject the rest, which can be used to do scaling, if you have a cluster. With shares it would think it has all the CPU cores available and oversize the queue.

Re: Go, Containers, and the Linux Scheduler

#108

Post by ithkuil »

eloisant wrote: post #103
ithkuil wrote: post #101

Earlier quoted context omitted.

I don't set limits because I'm afraid of how a pod is going to affect other pods. I set limits because I don't want to get used to being able to tap on the excess CPU available because that's not guaranteed to be available. As the node fills up with more and more other pods, it's possible that a pod that was running just fine a moment ago is crawling to a halt. Limits allow me to simulate the same behavior and plan f…

That's why you use monitor and alerting, so you notice degraded performances before the pods is crawling to a halt. You need to do it anyway because a service might progressively need more resources as it's getting more traffic, even if you're not adding any other pod.

Sure you need monitoring and alerting and sure there are other reasons why you need to update your requests.

But having _neighbours_ affecting the behaviour of your workload is precisely what creates the kind of fatigue that then results in people claiming that it's hard to run k8s workloads. K8s is highly dynamical, pods can get scheduled on a node by chance sometimes and on some clusters; pagers will ring, incidents will be created for conditions that may solve themselves because of another deployment (possibly of another team) happening.

Overcommit/bursting is an advanced cost saving feature.

Let me say it again: splitting up a large machine into smaller parts that can use the unused capacity of other parts in order to reduce waste is an advanced feature!

The problem is that the request/limits feature is presented in the configuration spec and in the documentation in a deceptively simple way and we're tricked to think it's a basic feature.

Not all companies have ops teams that are well equipped to do more sophisticated things. My advice for those teams who cannot setup full automation around capacity management is to just not use this advanced features.

An alternative is to just use smaller dedicated nodes and (anti)affinity rules, so you always understand which pods go with which other pods. It's clunky but it's actually easier to reason about what's going to happen.

EDIT: typos

Re: Go, Containers, and the Linux Scheduler

#110

Post by jpolidor »

richdougherty wrote: post #86
yjftsjthsd-h wrote: post #37

Earlier quoted context omitted.

I'm fairly certain that that .net had to deal with it and Java had or still has a problem, I forget which. (Or did you mean runtimes like containerd?)

Supported in Java 10 (and backported to Java 8) since 2018. Not sure about .NET. - "The JVM has been modified to be aware that it is running in a Docker container and will extract container specific configuration information instead of querying the operating system. The information being extracted is the number of CPUs and total memory that have been allocated to the container." https://www.oracle.com/java/technologi…

Then in Java, if you don't set the limits, it gets the CPU from the VM via Runtime.getRuntime().availableProcessors()... this method returns the number of CPUs of the VM or the value set as CPU Quota. Starting from Java 11 the -XX:+PreferContainerQuotaForCPUCount is by default true. For Java Edit: or set -XX:-PreferContainerQuotaForCPUCount
Post reply on HN