Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go, Containers, and the Linux Scheduler

riverphillips.dev

Post reply on HN
Print view View original

11–20 of 143 posts

Re: Go, Containers, and the Linux Scheduler

#11

Post by dilyevsky »

Thaxll wrote: post #9
dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

People using Kubernetes don't tune or change those settings, it's up to the app to behave properly.

False. Kubernetes cpu request sets the shares, cpu limit sets the cfs quota

Re: Go, Containers, and the Linux Scheduler

#12

Post by Thaxll »

dilyevsky wrote: post #11
Thaxll wrote: post #9

Earlier quoted context omitted.

People using Kubernetes don't tune or change those settings, it's up to the app to behave properly.

False. Kubernetes cpu request sets the shares, cpu limit sets the cfs quota

You said to change docker flags. Anyway your post is irrelevant, the goal is to let know the runtime about how many posix threads should it use.

If you set request / limit to 1 core but you run on 64 cores node , then you runtime will see that which will bring performance down.

Re: Go, Containers, and the Linux Scheduler

#13

Post by riv991 »

dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

Hi I'm the blog author, thanks for the feedback

I'll try and clarify this. I think this is how the sympton presents but I should be clearer.

Re: Go, Containers, and the Linux Scheduler

#14

Post by mratsim »

dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

> Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforcement.

One caveat is that an application can detect when --cpu is used as I think it's using cpuset. When quota are used it cannot detect and more threads than necessary will likely be spawned

Re: Go, Containers, and the Linux Scheduler

#15

Post by cpuguy83 »

mratsim wrote: post #14
dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

> Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforcement. One caveat is that an application can detect when --cpu is used as I think it's using cpuset. When quota are used it cannot detect and more threads than necessary will likely be spawned

It is not using cpuset (there is a separate flag for this). --cpus tweaks the cfs quota based on the number of cpus on the system and the requested amount.

Re: Go, Containers, and the Linux Scheduler

#16

Post by dekhn »

The common problem I see across many languages is: applications detect machine cores by looking at /proc/cpuinfo. However, in a docker container (or other container technology), that file looks the same as the container host (listing all cores, regardless of how few have been assigned to the container).

I wondered for a while if docker could make a fake /proc/cpuinfo that apps could parse that just listed "docker cpus" allocated to the job, but upon further reflection, that probably wouldn't work for many reasons.

Re: Go, Containers, and the Linux Scheduler

#17

Post by cpuguy83 »

dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

> "Don't use Docker's --cpu flag and instead use"

This is rather strong language without any real qualifiers. It is definitely not "mostly useless". Shares and quotas are for different use-cases, that's all. Understand your use-case and choose accordingly.

Re: Go, Containers, and the Linux Scheduler

#18

Post by dilyevsky »

Thaxll wrote: post #12
dilyevsky wrote: post #11

Earlier quoted context omitted.

False. Kubernetes cpu request sets the shares, cpu limit sets the cfs quota

You said to change docker flags. Anyway your post is irrelevant, the goal is to let know the runtime about how many posix threads should it use. If you set request / limit to 1 core but you run on 64 cores node , then you runtime will see that which will bring performance down.

Original article is about docker. That’s the point of my comment - dont set cpu limit

Re: Go, Containers, and the Linux Scheduler

#19

Post by dilyevsky »

mratsim wrote: post #14
dilyevsky wrote: post #3

This is subtly incorrect - as far as Docker is concerned CFS cgroup extension has several knobs to tune - cfs_quota_us, cfs_period_us (typical default is 100ms not a second) and shares. When you set shares you get weighted proportional scheduling (but only when there's contention). The former two enforce strict quota. Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforceme…

> Don't use Docker's --cpu flag and instead use --cpu-shares to avoid (mostly useless) quota enforcement. One caveat is that an application can detect when --cpu is used as I think it's using cpuset. When quota are used it cannot detect and more threads than necessary will likely be spawned

—cpu sets the quota, there is is a —cpuset-cpu flag for cpuset and you can detect both by looking at the /sys/fs/cgroup

Re: Go, Containers, and the Linux Scheduler

#20

Post by riv991 »

dilyevsky wrote: post #18
Thaxll wrote: post #12

Earlier quoted context omitted.

You said to change docker flags. Anyway your post is irrelevant, the goal is to let know the runtime about how many posix threads should it use. If you set request / limit to 1 core but you run on 64 cores node , then you runtime will see that which will bring performance down.

Original article is about docker. That’s the point of my comment - dont set cpu limit

I intended it to be applicable to all containerised environments. Docker is just easiest on my local machine.

I still believe it's best to set these variables regardless of cpu limits and/or cpu shares

Post reply on HN