Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux Namespaces and Go Don't Mix

weave.works

Post reply on HN
Print view View original

71–80 of 175 posts

Re: Linux Namespaces and Go Don't Mix

#71

Post by ahoka »

rkeene2 wrote: post #19
tankenmate wrote: post #6

Earlier quoted context omitted.

Because C++ is vastly more complex than either C or Go.

To go along with this, due to overloading and inheritance rules you cannot look at a small section of C++ code and determine what it does without reading up to the entire program.

I think that holds for any program written in any language. If the abstractions are confusing then it will be hard to read, be it layers of C macros, over-use of template programming or bad class hierarchy. BTW C++'s standard library is mostly functional (having roots in scheme), which makes it pretty easy to reason about.

Re: Linux Namespaces and Go Don't Mix

#72

Post by di4na »

pcwalton wrote: post #3
SwellJoe wrote: post #2

This leads to go code being roughly as messy/clumsy as C (or whatever else) code in the sections that need concurrency and also need to change namespace. That's unfortunate, but I don't know that it really "raises a few eyebrows". I mean, what's the better alternative to Go for this work? Maybe Rust? It is, at least more controllable at a lower level...but, not as easy to pick up for people coming from a C/Python/Per…

No, it's a lot messier than C. In a regular C program (not using any special libraries for M:N threading) you wouldn't have to spawn an entirely separate process. This issue is one of the downsides of Go's M:N scheduling. The OS is simply not aware of what the Go runtime is doing, and as a result you get impedance mismatches like this. It "raises a few eyebrows" because M:N scheduling is unpopular outside of Go and E…

Just saying, but Erlang do not target that type of "System Programming" and the answer to that problem in Erlang would probably work through totally different way to do it. This namespace thing would not be a problem.

This is not a problem of M:N. This is a problem of Go being badly designed. Not new.

Re: Linux Namespaces and Go Don't Mix

#73

Post by ieRei6ae »

holydude wrote: post #11
scriptkiddy wrote: post #4

Earlier quoted context omitted.

> I mean, what's the better alternative to Go for this work? Maybe Rust? It is, at least more controllable at a lower level...but, not as easy to pick up for people coming from a C/Python/Perl/Ruby systems and ops background. You know, it's interesting. I've been programming with Python for about 6 years now. I've also picked up Javascript, SQL, bash, and PHP along the way. I'm always gaining a little bit of C knowle…

Yes but the problem with languages like Nim is lack of support and maturity. Go is more versatile and at the same time more mature than anything out there. It is a different design and it excels in what it does (considering all tradeoffs now). Will Nim be as versatile and solid as Go in the future? Hard to predict but i would say no. You need a solid financial backing and certain amount of adoption where people actua…

> the problem with languages like Nim is lack of support and maturity

It was the same for Linux, Python, Ruby. Being community-driven can be a bug or a feature.

> Go is more versatile

Nim has macros, templates, overloading and compiles to C, JS, Objective-C. Runs on more architectures than Go including arduino microcontrollers.

> mature than anything out there

Go is not more mature than C, C++, Java, Python, Perl...

> You need a solid financial backing

See Linux, Python, Ruby... many projects had no big company or funding behind them.

Re: Linux Namespaces and Go Don't Mix

#74

Post by klodolph »

jankedeen wrote: post #70
cyphar wrote: post #20

Earlier quoted context omitted.

> C concurrency has been implemented with fork. Arguably it has been implemented with clone(2) which has flags. > in fact, it's easier to reason about fork than something like POSIX threads, IMHO. Not if you call fork() in a multi-threaded program. That ends _exceptionally_ badly (let's just say there's a reason Go doesn't expose syscall.Fork and it has to do with horrific deadlocks). > I just didn't see the problem…

No. You can call fork pretty cleanly in a pthread based program. This post is not informed criticism. There is some prep for ugly contingencies(pthread_atfork) but it usually just works. Namespace based code while doing the same is like setting your hair on fire while running through the gasoline forest..but caveat emptor is usually spelled out pretty critically in the API and docs. Oh, btw: Fuck RUST. For once and f…

> There is some prep for ugly contingencies(pthread_atfork) but it usually just works.

Actually, that's not true. Perhaps it's theoretically possible, in some cases, to make your threaded code work with fork with the appropriate pthread_atfork() handlers, but in general, it's a total mess. Thread A has a lock, thread B calls fork(), thread B tries to acquire the lock. The lock is held, but thread A is not running in the child. That much should be obvious.

The usual fix is to write a pthread_atfork() handler for every lock in your program. Before forking, it locks all of your mutexes. After forking, in both the child and parent, it unlocks them. You can see the holes in this. One simple hole is that fork() is async signal safe but pthread_mutex_lock() is not, but most people won't fork() in a signal anyway. The big problem is that you now need to make all of your locks recursive and figure out, globally, what order to lock them in. This means registering your atfork() handlers in exactly the right order, something which is more difficult than it sounds. Another problem is the performance implications of acquiring all the locks in your program.

Namespaces + threads are fine, the only problem here is namespaces + M:N threads + no method for pinning a green thread to an OS thread, like forkOS in Haskell.

Re: Linux Namespaces and Go Don't Mix

#75

Post by mrjana »

vishvananda wrote: post #13

Author of the go netlink library here. I've run into this issue a number of times. There has been conversation in the past about adding some kind of new runtime command like LockOSThread to prevent new threads from being spawned, but it didn't gain any momentum. Even though I am a big fan of go, I've personally built two container runtimes in other languages do to the namespace clumsiness. Personally, I think rust is…

One way to alleviate the problem at least for netlink library is to create a function which calls runtimeLockOSThread, sets into the required namespace and then opens a netlink socket using only raw syscall apis. One has to be careful in this code path to not invoke go runtime (i.e both socket and setns should be raw syscall apis) and not even trigger any allocations so that go runtime doesn't get a chance a spin a new OS thread. Once a socket is created in the required namespace you can get back to the caller namespace and return the socket fd. Now this socket fd is bound to that namespace and all netlink operations on that socket will happen in the target namespace.

Disclaimer: I am one of the original libnetwork authors and we have been aware of this issue with go for some time now.

Re: Linux Namespaces and Go Don't Mix

#76

Post by unscaled »

zellyn wrote: post #63
scriptkiddy wrote: post #54

Earlier quoted context omitted.

No, I encourage you to shill in fact. > The reasons Go doesn't (yet) have generics are practical rather than philosophical. And well-documented. Can you give an example? I guess I fail to understand why a statically typed language would choose to forgo all of the advantages generics provide. Does it have something to do with Goroutines? > I also don't believe Google's marketing budget contributed much if anything to…

> Can you give an example? I guess I fail to understand why a statically typed language would choose to forgo all of the advantages generics provide. Does it have something to do with Goroutines? Like I said, the objections are practical, not philosophical: They published four past design docs for generics in Go that simply didn't pass technical muster ( https://github.com/golang/proposal/blob/master/design/15292-...…

> Like I said, the objections are practical, not philosophical

Russ Cox and Rob Pike keep repeating this mantra, but I don't buy it. No, I don't believe they object the idea of generics itself. And yes, generics pose practical complexities and a plethora of issues that have to be resolved.

But so does every other language feature. The features you choose to add reflect your philosophical values and priorities.

Go chose to bake some hitherto very niche features that could have been as libraries into the language. Channels in particular, are a language construct only because Go doesn't allow operator overloading and generics for user types. But channels get them, because channels are demonstratively important for Rob Pike[1].

There's nothing wrong with that of course, but that's a philosophical decision. Why channels can avoid the vagaries of interface{} boxing, but not sets, linked lists or queues?

The reality is that every modern statically typed language except Go has generics, and they all implemented them very well. The ML languages and Ada were already doing it in the 80s, and OCaml and Eiffel managed to combine generics and polymorphism back in the 90s.

It seems to me that when originally designing Go, Pike, Griesemer and Thompson just didn't think generics are worthwhile enough for the effort it takes to properly research them.

When Go was started, its authors mostly looked to languages which implemented generics later in their lifecycle (namely Java and C++) and their implementations suffered from problems due to other, rather obvious, design defects: https://research.swtch.com/generic

I'm happy to see that this attitude is changing, and other languages are looked at.

[1] https://swtch.com/~rsc/thread/

Re: Linux Namespaces and Go Don't Mix

#77

Post by steveklabnik »

jankedeen wrote: post #70
cyphar wrote: post #20

Earlier quoted context omitted.

> C concurrency has been implemented with fork. Arguably it has been implemented with clone(2) which has flags. > in fact, it's easier to reason about fork than something like POSIX threads, IMHO. Not if you call fork() in a multi-threaded program. That ends _exceptionally_ badly (let's just say there's a reason Go doesn't expose syscall.Fork and it has to do with horrific deadlocks). > I just didn't see the problem…

No. You can call fork pretty cleanly in a pthread based program. This post is not informed criticism. There is some prep for ugly contingencies(pthread_atfork) but it usually just works. Namespace based code while doing the same is like setting your hair on fire while running through the gasoline forest..but caveat emptor is usually spelled out pretty critically in the API and docs. Oh, btw: Fuck RUST. For once and f…

> Oh, btw: Fuck RUST. For once and for all fuck this tyranny of coercion by potential IP holders. It is a shit language.

... what? I am confused as to what you're saying here.

Re: Linux Namespaces and Go Don't Mix

#78

Post by jankedeen »

klodolph wrote: post #74
jankedeen wrote: post #70

Earlier quoted context omitted.

No. You can call fork pretty cleanly in a pthread based program. This post is not informed criticism. There is some prep for ugly contingencies(pthread_atfork) but it usually just works. Namespace based code while doing the same is like setting your hair on fire while running through the gasoline forest..but caveat emptor is usually spelled out pretty critically in the API and docs. Oh, btw: Fuck RUST. For once and f…

> There is some prep for ugly contingencies(pthread_atfork) but it usually just works. Actually, that's not true. Perhaps it's theoretically possible, in some cases, to make your threaded code work with fork with the appropriate pthread_atfork() handlers, but in general, it's a total mess. Thread A has a lock, thread B calls fork(), thread B tries to acquire the lock. The lock is held, but thread A is not running in…

*...it's a total mess...theoretically it's possible....speculation on standard behavior based on platform behavior...namespace(x)chosen implementation detail...functional language reference... -- Translation: Design is broken

Re: Linux Namespaces and Go Don't Mix

#79

Post by SwellJoe »

fmstephe wrote: post #62
scriptkiddy wrote: post #56

Earlier quoted context omitted.

Well, Go is obviously not a bad language. That said, I do think that it's adoption was primarily fueled by Google's popularity. Imagine go being released by a single person or small group of people. How would people have heard about it?

I think this isn't quite true. From my perspective, Go did get a massive boost but not from Google. The language was originally designed by Robert Griesemer, Rob Pike, and Ken Thompson. It is fair to say that without those names attached I would have likely passed it by. Google is boring, but those three names, for me at least, I had to take a look.

If Google could make a language popular, where does Dart fit into the story? People seem to love Go, and with very little encouragement from Google. I don't see a lot of love for Dart...though everyone who's played with it seems to like it well enough.

Re: Linux Namespaces and Go Don't Mix

#80

Post by zellyn »

unscaled wrote: post #76
zellyn wrote: post #63

Earlier quoted context omitted.

> Can you give an example? I guess I fail to understand why a statically typed language would choose to forgo all of the advantages generics provide. Does it have something to do with Goroutines? Like I said, the objections are practical, not philosophical: They published four past design docs for generics in Go that simply didn't pass technical muster ( https://github.com/golang/proposal/blob/master/design/15292-...…

> Like I said, the objections are practical, not philosophical Russ Cox and Rob Pike keep repeating this mantra, but I don't buy it. No, I don't believe they object the idea of generics itself. And yes, generics pose practical complexities and a plethora of issues that have to be resolved. But so does every other language feature. The features you choose to add reflect your philosophical values and priorities. Go cho…

> The reality is that every modern statically typed language except Go has generics, and they all implemented them very well. The ML languages and Ada were already doing it in the 80s, and OCaml and Eiffel managed to combine generics and polymorphism back in the 90s.

> It seems to me that when originally designing Go, Pike, Griesemer and Thompson just didn't think generics are worthwhile enough for the effort it takes to properly research them.

Are you arguing that it's easy, and they just haven't done it? Or are you arguing that it wasn't important enough at first, and so the implementation developed in directions that preclude straightforward implementations now? I can believe the latter, but after reading the multiple, detailed proposals from Ian Lance Taylor, I don't believe the former.

Post reply on HN