Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go, Containers, and the Linux Scheduler

riverphillips.dev

Post reply on HN
Print view View original

71–80 of 143 posts

Re: Go, Containers, and the Linux Scheduler

#71

Post by cedws »

marcus_holmes wrote: post #60

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't…

A lot of companies are Kubernetes based now so containers are the default delivery mechanism for all software.

Re: Go, Containers, and the Linux Scheduler

#72

Post by [deleted] »

marcus_holmes wrote: post #60

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't…

[deleted]

Re: Go, Containers, and the Linux Scheduler

#73

Post by tail_exchange »

jeffbee wrote: post #67
tail_exchange wrote: post #65

Earlier quoted context omitted.

Containers can give you better isolation between the application and the host, as well as making horizontal scaling easier. It's also a must if you are using a system like Kubernetes. If you are running multiple applications in the same host, you also have control over resource limits.

Kubernetes requires a platform runtime that answers its requests, but there's no law enforcement agency that will prevent you from using a custom runtime that ignores the very existence of Linux control groups.

Yes, that is true. Though if you are using Google's or Amazon's managed Kubernetes services, I think you need to use Docker.

Re: Go, Containers, and the Linux Scheduler

#74

Post by marcus_holmes »

throwaway892238 wrote: post #70
marcus_holmes wrote: post #60

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't…

Here's what a container gives you: - Isolation of networking - Isolation of process namespace - Isolation of filesystem namespace - CPU and Memory limit enforcement - A near-universal format for packaging, distributing, and running applications, with metadata, and support for multiple architectures - A simple method for declaring a multi-step build process - Other stuff I don't remember You're certainly welcome to go…

A VM provides the first 4 anyway - if you're deploying to a cloud instance then having these in the container is redundant. If you're deploying to bare metal then it's possibly useful, but only if you're deploying multiple containers to the same machine.

Go doesn't need a format for packaging - it's one file. It's becoming common practice to embed everything else into the binary. (side note: I haven't done this with env files yet, and tend to deploy them separately, but I don't see any reason why we don't do this and produce binaries targetted at the specific deployment environments. I might give it a go).

I kinda prefer makefiles for the build stuff, or even just a script. The whole process of creating a Docker instance, pushing source files to it, trigging go build and then pulling back the binary seems redundant; there's no advantage of doing this in a container over doing it on the local machine. And it's a lot faster on the local machine.

Talking to people, it appears to be as mholt said: everyone just does everything in containers so apparently we do this too.

Re: Go, Containers, and the Linux Scheduler

#75

Post by marcus_holmes »

mholt wrote: post #66
marcus_holmes wrote: post #60

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't…

As the author of Caddy, I have often wondered why people run it in containers. The feedback I keep hearing is basically workflow/ecosystem lock-in. Everything else uses containers, so the stuff that doesn't need containers needs them now, too.

Love your work :) Good to know I'm not totally out there on this :)

Re: Go, Containers, and the Linux Scheduler

#76

Post by ecnahc515 »

pbh101 wrote: post #61
vbezhenar wrote: post #48

Earlier quoted context omitted.

I don't think that's correct. --cpus is the same as --cpu-period which is cpu limit. You can easily check it yourself, just run docker container with --cpus set, run multi-core load there and check your activity monitor.

CFS quotas only become active under contention and even then are relative: if you’re the only thing running on the box and want all the cores but only set one cpu, you get all of them anyway. If you set cpus to 2 and another process sets to 1 and you both try to use all CPUs all out, you’ll get 66% and they’ll get 33%. This isn’t the same as cpusets, which work differently.

> CFS quotas only become active under contention

That's not true at all. Take a look at `cpu.cfs_quota_us` in https://kernel.googlesource.com/pub/scm/linux/kernel/git/glo...

It's a hard time limit. It doesn't care about contention at all.

`cpu.shares` is relative, for choosing which process gets scheduled, and how often, but the CFS quota is a hard limit on runtime.

Re: Go, Containers, and the Linux Scheduler

#77

Post by marcus_holmes »

wahern wrote: post #69
marcus_holmes wrote: post #60

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't…

Go programs still make use of, e.g., /etc/resolv.conf and /etc/ssl/certs/ca-certificates.crt. These aren't strictly necessary, but using them makes it easier--more consistent, more transparent--to configure these resource across different languages and frameworks.

I use the semi-official Acme package [0], which handles all that really well. I haven't touched SSL or TLS config for years. I mean, I might be an outlier, but this seems pretty standard for Go deployments these days.

[0] https://pkg.go.dev/golang.org/x/crypto/acme

Re: Go, Containers, and the Linux Scheduler

#78

Post by n3t »

jeffbee wrote: post #54
n3t wrote: post #47

Earlier quoted context omitted.

> which of problematic in K8s where the visible CPUs may change while your process runs This is new to me. What is this… behavior? What keywords should I use to find any details about it? The only thing that rings a bell is requests/limit parameters of a pod but you can't change them on an existing pod AFAIK.

If you have one pod that has Burstable QoS, perhaps because it has a request and not a limit, its CPU mask will be populated by every CPU on the box, less one for the Kubelet and other node services, less all the CPUs requested by pods with Guaranteed QoS. Pods with Guaranteed QoS will have exactly the number of CPUs they asked for, no more or less, and consequently their GOMAXPROCS is consistent. Everyone else will…

If by "CPU mask" you refer to the `sched_getaffinity` syscall, I can't reproduce this behavior.

What I tried: I created a "Burstable" Pod and run `nproc` [0] on it. It returned N CPUs (N > 1).

Then I created a "Guaranteed QoS" Pod with both requests and limit set to 1 CPU. `nproc` returned N CPUs on it.

I went back to the "Burstable" Pod. It returned N.

I created a fresh "Burstable" Pod and run `nproc` on it, got N again. Please note that the "Guaranteed QoS" Pod is still running.

> Pods with Guaranteed QoS will have exactly the number of CPUs they asked for, no more or less

Well, in my case I asked for 1 CPU and got more, i.e. N CPUs.

Also, please note that Pods might ask for fractional CPUs.

[0]: coreutils `nproc` program uses `sched_getaffinity` syscall under the hood, at least on my system. I've just checked it with `strace` to be sure.

Re: Go, Containers, and the Linux Scheduler

#79

Post by fn-mote »

telotortium wrote: post #52
wutwutwat wrote: post #49

Earlier quoted context omitted.

I only use `nproc` and see it used in other containers as well, ie `bundle install -j $(nproc)`. This honors cpu assignment and provides the functionality you're seeking. Whether or not random application software uses nproc if available, idk > Print the number of processing units available to the current process, which may be less than the number of online processors. If this information is not accessible, then prin…

This is not very robust. You probably should use the cgroup cpu limits where present, since `docker --cpus` uses a different way to set quota: if [[ -e /sys/fs/cgroup/cpu/cpu.cfs_quota_us ]] && [[ -e /sys/fs/cgroup/cpu/cpu.cfs_period_us ]]; then GOMAXPROCS=$(perl -e 'use POSIX; printf "%d\n", ceil($ARGV[0] / $ARGV[1])' "$(cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us)" "$(cat /sys/fs/cgroup/cpu/cpu.cfs_period_us)") else GO…

A shell script that invokes perl to set an environment variable used by Go. Some days I feel like there is a lot of duct tape involved in these applications.

Re: Go, Containers, and the Linux Scheduler

#80

Post by jeffbee »

n3t wrote: post #78
jeffbee wrote: post #54

Earlier quoted context omitted.

If you have one pod that has Burstable QoS, perhaps because it has a request and not a limit, its CPU mask will be populated by every CPU on the box, less one for the Kubelet and other node services, less all the CPUs requested by pods with Guaranteed QoS. Pods with Guaranteed QoS will have exactly the number of CPUs they asked for, no more or less, and consequently their GOMAXPROCS is consistent. Everyone else will…

If by "CPU mask" you refer to the `sched_getaffinity` syscall, I can't reproduce this behavior. What I tried: I created a "Burstable" Pod and run `nproc` [0] on it. It returned N CPUs (N > 1). Then I created a "Guaranteed QoS" Pod with both requests and limit set to 1 CPU. `nproc` returned N CPUs on it. I went back to the "Burstable" Pod. It returned N. I created a fresh "Burstable" Pod and run `nproc` on it, got N a…

I don't know what nproc does. Consider `taskset`
Post reply on HN