Linux Namespaces and Go Don't Mix • Hacker News Board

Re: Linux Namespaces and Go Don't Mix

#151

Post by bboreham » Sat, Jun 03, 2017, 2:05 PM UTC

the_mitsuhiko wrote: post #144

valarauca1 wrote: post #119

Earlier quoted context omitted.

>Go assumes that all threads are equal and therefore any Goroutine can run on any thread and threads are all equal This isn't true in Linux or Windows so I don't see why Go would make this assumption other than poor design. >The property should be accessible via a handle that can be shared amongst all threads. It literally is. The namespace information is shared with all threads in a PID group globally available in t…

It's not true on macOS either. Threads can have different working directories for instance.

I'd be interested to know how to do that. Go's Chdir() calls SYS_CHDIR on Darwin, and Posix requires that chdir affects the whole process.

In Linux you can unset CLONE_FS when creating a thread, but the Go runtime does not do this.

I note the Windows docs[1] say "Multithreaded applications and shared library code should not use the SetCurrentDirectory function" (which Go's Chdir() calls)

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

Re: Linux Namespaces and Go Don't Mix

#152

Post by bboreham » Sat, Jun 03, 2017, 2:08 PM UTC

YZF wrote: post #124

the8472 wrote: post #118

Earlier quoted context omitted.

But linux threads predate go, so go is built on assumptions they knew were not true.

It's possible they didn't know different Linux threads can be in different namespaces, it's not exactly something everyone knows. Namespaces in Linux aren't that new (15 years or so). It's possible they knew but decided this wasn't an important enough use case to warrant language features. I've never needed to have half my process in one namespace and half in another. It's a niche application. Green threads/goroutine…

https://github.com/golang/go/issues/1435

It's not exactly the same issue, but the themes are all the same.

Re: Linux Namespaces and Go Don't Mix

#153

Post by jchw » Sat, Jun 03, 2017, 2:27 PM UTC

jeremyjh wrote: post #149

bhawks wrote: post #135

I think the proper title is: Linux Process and Threads Don't Mix. The Linux syscall interface exposes certain functionalities that are much more easy to reason about at the process level such as namespaces, capabilities, seteuid and so on. However these syscalls all operate on the thread level (since the kernel treats threads pretty similarly to processes). Therefore in order to perform these operations safely you ne…

The M:N problem in Go is that you cannot control which thread runs which code. So yes, you could wish the OS exposed different APIs, but presently you can manage this situation in languages that let you manage threads.

The person you're replying to has already made that clear. It's, in fact, also possible to manage the problem in Go with some finagling. But like Go, if you have multiple threads, it's difficult

I've hit this exact problem with multithreading in C and setuid and just because it _can_ be managed in C doesn't make it easy or straightforward.

Therefore, I mirror the sentiment that there needs to be a way to operate on a process level, even if that has some interesting consequences.

(P.S.: In C, if you're using glibc, it DOES actually patch this issue up on its own using one hell of a nasty hack.)

Re: Linux Namespaces and Go Don't Mix

#154

Post by lunixbochs » Sat, Jun 03, 2017, 3:24 PM UTC

vbernat wrote: post #48

lunixbochs wrote: post #46

Earlier quoted context omitted.

> The problem is Go runtime does not provide a way to exclusively lock a system thread from existing or future routines. runtime.LockOSThread() does exactly that [1]. [1] https://golang.org/pkg/runtime/#LockOSThread

No, it doesn't. Go runtime can decide to spawn a new thread from the locked one. The new thread will inherit the characteristics (namespace, uid) of the old one. See https://github.com/docker/libnetwork/issues/1113 for more details.

> Go runtime can decide to spawn a new thread from the locked one

That sounds like an implementation issue, why not assume the documentation is the intended behavior and this side effect is a bug? I'd support a CL to fix this behavior or add a block-clone-from-here runtime call (but the end result of that is you want the thread to exit when you're done with it, and not to go back into the pool... which is also new behavior). At the minimum, this behavior of new threads spawning from LockOSThread could be documented.

As a workaround, what about CGO -> pthreads -> spawn a control thread free from the Go scheduler -> call back into Golang to run a control loop function? You can do this in init() to ensure it has full control over itself. Or will Golang call clone() from unscheduled code?

Re: Linux Namespaces and Go Don't Mix

#155

Post by lunixbochs » Sat, Jun 03, 2017, 3:31 PM UTC

cyphar wrote: post #131

lunixbochs wrote: post #46

Earlier quoted context omitted.

> The problem is Go runtime does not provide a way to exclusively lock a system thread from existing or future routines. runtime.LockOSThread() does exactly that [1]. [1] https://golang.org/pkg/runtime/#LockOSThread

The documentation for both GOMAXPROCS and runtime.LockOSThread lie to you. Neither of them allow you to stop the runtime from creating new threads (or executing library code in the wrong thread). Trust me, we've been trying to get Go to co-operate with containers for quite a few years. It's not as simple as just reading the standard library docs. ;)

Okay, I think I understand. I thought the problem referred to above with setuid was separate and would be fixed by locking the thread, but spawning sub-threads with the wrong UID is a problem. See also my sibling response.

Re: Linux Namespaces and Go Don't Mix

#156

Post by sly010 » Sat, Jun 03, 2017, 3:50 PM UTC

SwellJoe wrote: post #23

cyphar wrote: post #12

Earlier quoted context omitted.

Rust is much better for this (though I still feel some of the fork/exec interface has similar warts to Go). However, you're wrong in saying that it's "roughly as messy/clumsy" as C. Let me tell you how runc works. runc is written in Go, and we take an OCI configuration file. Because we can't just fork and set up all of the namespaces in Go, we have a C function called nsexec which is specified as __attribute__((const…

Thanks for explaining how it's done in runc. That does sound pretty awful. So, even though the initialization can be outsourced to a C function, you still would prefer to be working entirely in C? Are there no advantages to Go for runc? And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work? Is it merely fear of C that keeps so much of the container infrastructure…

> Is it merely fear of C that keeps so much of the container infrastructure on Go?

Sort of. Go binaries are statically linked by default which is a must in situations where you are e.g. unsure about what libs are available in the current environment. You have to go through a lot of hoops to make sure your C executable is really fully statically linked.

Re: Linux Namespaces and Go Don't Mix

#157

Post by je42 » Sat, Jun 03, 2017, 4:58 PM UTC

cyphar wrote: post #136

jankedeen wrote: post #81

Earlier quoted context omitted.

* Coercion: large down vote/propaganda community against any negative rust idea..evidenced here. * IP Holders, well this is speculative. I can't understand anyone who would compare Rust against C in a positive way without possibly profiting from it. Rust is painful to write, painful to learn and makes you ask yourself: Can I not write a well designed and correct program in C? Of course I can. Why Am I Learning Rust?…

> Rust is painful to write, painful to learn and makes you ask yourself: It's painful to write because writing software that is memory safe is not easy. Rust makes it insanely easy in comparison to the "old way" of doing it (hacking together a C program then having to patch security holes every release, never being certain that your code is actually memory safe). > Can I not write a well designed and correct program…

also, one key aspect is. There are may be individuals out there that can do this.

however, usually code is written by multiple people. (and i would also count your future self as another person).

that collaboration on the same code base requires automatic tests, otherwise the next person will break the code in non-obvious ways.

Re: Linux Namespaces and Go Don't Mix

#158

Post by the_mitsuhiko » Sat, Jun 03, 2017, 5:45 PM UTC

bboreham wrote: post #151

the_mitsuhiko wrote: post #144

Earlier quoted context omitted.

It's not true on macOS either. Threads can have different working directories for instance.

I'd be interested to know how to do that. Go's Chdir() calls SYS_CHDIR on Darwin, and Posix requires that chdir affects the whole process. In Linux you can unset CLONE_FS when creating a thread, but the Go runtime does not do this. I note the Windows docs[1] say "Multithreaded applications and shared library code should not use the SetCurrentDirectory function" (which Go's Chdir() calls) [1] https://msdn.microsoft.co…

SYS___pthread_chdir exists on mac os.

Re: Linux Namespaces and Go Don't Mix

#159

Post by the_mitsuhiko » Sat, Jun 03, 2017, 5:51 PM UTC

bhawks wrote: post #135

I think the proper title is: Linux Process and Threads Don't Mix. The Linux syscall interface exposes certain functionalities that are much more easy to reason about at the process level such as namespaces, capabilities, seteuid and so on. However these syscalls all operate on the thread level (since the kernel treats threads pretty similarly to processes). Therefore in order to perform these operations safely you ne…

> The Linux syscall interface exposes certain functionalities that are much more easy to reason about at the process level such as namespaces, capabilities, seteuid and so on. However these syscalls all operate on the thread level (since the kernel treats threads pretty similarly to processes). Therefore in order to perform these operations safely you need some sort of process wide mechanism to apply the operation on every thread (and don't forget error handling!)

This is hardly a linux specific issue. Prominently for instance pthread_setugid_np exists on OS X, threads for different subsystems exist on Windows etc.

Re: Linux Namespaces and Go Don't Mix

#160

Post by SwellJoe » Sat, Jun 03, 2017, 6:58 PM UTC

cyphar wrote: post #127

SwellJoe wrote: post #23

Earlier quoted context omitted.

Thanks for explaining how it's done in runc. That does sound pretty awful. So, even though the initialization can be outsourced to a C function, you still would prefer to be working entirely in C? Are there no advantages to Go for runc? And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work? Is it merely fear of C that keeps so much of the container infrastructure…

> you still would prefer to be working entirely in C? No, but there are more options than just Go and C. Rust is an option that I'm shilling at the moment (though I've only started learning it, so take that with a grain of salt). The main reason I would want to write it in C is because there's a lot of string parsing code you have to write in order to make container runtimes work -- and as we all know that's probably…

Thanks so much for the in-depth response!

I've been tinkering with go, and the project I'm planning to use it for is container-oriented (building/using/distributing them for non-technical users, more than working with them at a very low level like runC or similar, but still, it's very useful to know). As an aside, umoci looks, at first glance, like one of the components I thought I'd need to build...so, that's cool.

I'm not gonna keep bugging you with questions; I'll go read the code. (I'm also finding rust really neat, conceptually, even if I'm not yet finding it easy to read or understand.)