Live data from Hacker News

The case of a leaky goroutine

brainbaking.com

11–20 of 87 posts

Re: The case of a leaky goroutine

#11
post #2

It's a pity Go didn't have structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g... There's a library for it: https://github.com/sourcegraph/conc But this goes to one of the things I've been kind of banging on about languages, which is that if it's not in the language, or at least the standard library right at the beginning, sometimes it almost might as well not exist. Sometimes a new l…

[deleted]

Re: The case of a leaky goroutine

#12
post #8
post #2

It's a pity Go didn't have structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g... There's a library for it: https://github.com/sourcegraph/conc But this goes to one of the things I've been kind of banging on about languages, which is that if it's not in the language, or at least the standard library right at the beginning, sometimes it almost might as well not exist. Sometimes a new l…

> The concept was formulated in 2016 by Martin Sústrik (creator of ZeroMQ) with his C library libdill, with goroutines as a starting point. It's fairly new; the thing (and I think you address it too) is that the pattern did not exist yet when Go was introduced. Go is averse to adding more things to its standard library, or indeed changing its core fundamentals; I think it's better to have one well-defined way of doin…

> It's fairly new

That specific formulation is new but the concept had been floating around for a while. For instance Rust originally got scoped thread in 2015 (before they had to be removed for being unsound).

Re: The case of a leaky goroutine

#13
post #3

Thread leaks, which happen more frequently due to threaded async abstractions, such as the goroutine, are less often discussed than memory or CPU leaks, but are much more dangerous in multi-tenant container environments. A thread leak can lock up your entire node, including all the control plane processes. A container spec doesn't provide an easy way to control thread/nproc/ulimit limits (you can still do it, but it'…

Isn't this addressed by CPU requests/limits + pid limiting?

https://kubernetes.io/docs/concepts/policy/pid-limiting/

Re: The case of a leaky goroutine

#14
post #2

It's a pity Go didn't have structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g... There's a library for it: https://github.com/sourcegraph/conc But this goes to one of the things I've been kind of banging on about languages, which is that if it's not in the language, or at least the standard library right at the beginning, sometimes it almost might as well not exist. Sometimes a new l…

Guy Steele's talk "Growing A Language" gets into this. It's definitely worth looking up if you haven't seen it.

I think we've seen two ways languages have successfully been open to evolution:

Java was specifically designed to allow it to build on the standard library over time in fully backwards-compatible ways. It has required the central governance committee to adopt proposals, because reflection is slow and poorly-optimized, but it is a far more fully-featured language today than it was an inception without ever needing a reset. By keeping the surface area small & the strong "Everything Is An Object" paradigm in place, it has had remarkable longevity and has avoided the Python versioning pain.

The second are the "sharp knives" languages: Javascript, Ruby and to a lesser extent C++ (only because DLL hell is very real).

All three of these can be used to write software in any paradigm (including Aspects, if one is a masochist), and so let engineers invest in their own productivity. Languages where the standard libraries are indistinguishable from custom libraries require more skill and collective team alignment to use productively and safely, but also allow for solutions highly-opinionated languages can't support.

Re: The case of a leaky goroutine

#15
post #8
post #2

It's a pity Go didn't have structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g... There's a library for it: https://github.com/sourcegraph/conc But this goes to one of the things I've been kind of banging on about languages, which is that if it's not in the language, or at least the standard library right at the beginning, sometimes it almost might as well not exist. Sometimes a new l…

> The concept was formulated in 2016 by Martin Sústrik (creator of ZeroMQ) with his C library libdill, with goroutines as a starting point. It's fairly new; the thing (and I think you address it too) is that the pattern did not exist yet when Go was introduced. Go is averse to adding more things to its standard library, or indeed changing its core fundamentals; I think it's better to have one well-defined way of doin…

Yes, I do know Go predates a solid description of the paradigm, though I hid that in what otherwise looks like a bizarre verb tense in my first sentence. :)

Part of the reason I rhapsodize about new languages off of that observation is precisely that Go can't add it. It almost wouldn't even matter if they tried to put it in the language proper, because by backwards compatibility the old ways would still work, and it would take a very long time to get the entire ecosystem to the new way.

Re: The case of a leaky goroutine

#16
post #7
post #4

Earlier quoted context omitted.

My #1 complaint about Rust is that leaking a future is safe. It means the compiler can’t check for async coroutine leaks, and it breaks the borrow checker’s ability to say “nothing else has a reference to this any more”. Anyway, we’re using golang for some stuff at work, and holy crap, I forgot how terrible it was to work in high level languages that don’t statically check for correct synchronization. If C++-style co…

What is a "correct synchronization"?

Synchronization such as the whole program is well-defined according to some memory model.

Re: The case of a leaky goroutine

#17
If Go allowed something like "handle = go foo()", goroutine could be automatically terminated when handle goes out of scope or becomes dead and is garbage collected. You can also use handle to cancel a goroutine etc.

Go designers specifically avoided this model (of having a goroutine "id") for reasons I don't remember any more (may be to avoid making them heavier weight?) but this would be one way to stop leaky goroutines.

Re: The case of a leaky goroutine

#18
The explanation of the issue in ToDoneInterface really is not clear to me because of this:

> The defer close() seems to close well, but it’s on the wrong channel.

The `done` input channel is supposed to be closed by a caller, and the goroutine is closing the output channel, surely that's the point?

Now from what I know of go channels and understand of the code involved, the `done` channel may never get closed by the parents (and can never get closed at all if it's nil?), in which case the goroutine never receives a signal, never terminates, and leaks. But the explanation below the snippet confuses me completely.

And if that's that... what's the fix? Aside from not doing this sort of conversions? Just "git good scrub", try to make sure you don't rely on cancellation for progress, and hope you don't use raw background channels again, and don't forget to cancel your non-background channels?

Re: The case of a leaky goroutine

#20
A lot of these problems come from accepting a line in the OP, "A Goroutine is essentially a coroutine". The rest of the sentence is "...that maps onto green threads that map onto real native threads on your OS in an NxM way".

This is not a coroutine at all, calling them goroutines was a clever hacker pun along the lines of "GNU's Not Unix". If you treat a preëmptively-scheduled primitive as though it's a cooperatively-scheduled primitive, you're going to have a bad time.

Goroutines are threads, basically, with all the memory-management headaches that implies. Caveat emptor.

Post reply on HN