Linux Namespaces and Go Don't Mix

If you have one thread in one namespace and another in another you now have to worry about what you can do in the context of a callback. This asymmetry just makes any multi-threaded program more complicated than it needs to be (and already is).

C++ looks good on paper. It has containers and types and generics. You come to find out that it's all based on meta-programming which is an interesting idea: Basically you're writing code to write code, and that code writes code. The whole system is macros all the way down. There's no native language support for anything but macros and the macros implement everything.

The student's experience is that he can quickly solve a problem using a list of stacks of strings (vector > > in the parlance.) Which is fine, almost like typed-python until you make a mistake. Then, the compiler, who knows nothing about those types which were all built by expanding macros, is not your friend.

Miss a minor `*` and a single line of code will fully expand its underlying macros giving a 10-page, indecipherable error message.

Should you make it to runtime, no debugger can tell you the contents of a vector, string or stack. They're just blobs of buffers and pointers with mangled (and yeah, that's the word that they chose: mangled) names to make them extra unreadable.

This is when most people ask if they can just have C back.

But there also are non-security applications of namespaces.

And it's not like namespaces are the only per-thread thing in linux. Capabilities, uid and signal handlers come to mind.

No, but there are more options than just Go and C. Rust is an option that I'm shilling at the moment (though I've only started learning it, so take that with a grain of salt). The main reason I would want to write it in C is because there's a lot of string parsing code you have to write in order to make container runtimes work -- and as we all know that's probably the #1 source of security vulnerabilities.

> Are there no advantages to Go for runc?

There are, mainly due to network effect (everything else is written in Go) and getting contributions from the community (Go is easy to pick up). Unfortunately there are also disadvantages, and quite a few of those disadvantages are present in Go but are not present in other memory-safe and low-level languages (Rust is a good example because to be quite honest it's the only player in this space that doesn't try to do more than necessary).

Go is a good language for it's designed for (Web servers and similar things), but from my experience it's not the best choice for low-level tasks. We've seen cases where long-running container daemons (not naming any names) will crash if you run more than 1000 containers on a single system. They don't crash because of the actual daemon code but because of issues with Go's GC (it doesn't actually free memory sanely, it uses MADV_DONTNEED which inflates RSS and causes OOM to kill your daemon).

> And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?

Of course (and you could argue that we have done that in runc with github.com/opencontainers/runc/libcontainer/nsenter), but the thing to note is that in order to get around problems in the Go runtime you have to split out a single piece of code into separate processes and have to now redesign how a single function would work. So moving it to a separate library means that development is even more frustrating (you've created an API around the internal implementation of whatever thing you're working on).

> Is it merely fear of C that keeps so much of the container infrastructure on Go?

I think the network effect is the main reason. Most of the people I've worked with know quite a lot of C (we do kernel work sometimes) so writing a runtime in C would be frustrating but entirely doable. The problem is that you couldn't just import it into a Go project (and people don't like cgo because it makes binaries harder to build in certain cases).

> I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C

Go definitely has it's uses, and I still use it for new projects. For example I recently wrote a tool for dealing with OCI container images in Go[1]. The standard library of Go is quite nice (though I found some bugs in archive/tar but let's not go there) and I'm always quite amazed just how much you can do before you have to import external libraries.

But I recently started learning Rust and I am _really_ enjoying being able to understand what my program will actually look like when compiled. If you've ever had to strace a Go program, you'll know exactly what I mean. Debugging Go programs is basically fucking impossible.

[1]: https://github.com/openSUSE/umoci

Oh, and I haven't even mentioned the absolute shitfest that is the cgroup namespace and how you have to set up cgroups before you unshare it because its behaviour changes based on what cgroups you were in when you unshared it.