Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case of a leaky goroutine

brainbaking.com

Post reply on HN
Print view View original

61–70 of 87 posts

Re: The case of a leaky goroutine

#61

Post by pdimitar »

sapiogram wrote: post #55
Thaxll wrote: post #41

Earlier quoted context omitted.

So it's absent from every language but Rust?

Javascript is even better, by not having multithreading at all. I am not joking, Go is much worse than Java and C#, but Javascript and Rust are the only mainstream languages where I've seen non-experts reliably write correct concurrent code. Maybe it's true for langauges like Elixir as well, but I haven't tried.

It's 95% true for Elixir as well; it's very difficult to mess up parallelization or just leave a system in an incoherent state with it. Immutability and message passing help a lot.

Re: The case of a leaky goroutine

#62

Post by pdimitar »

treyd wrote: post #39
whateveracct wrote: post #36

People hate on Haskell async exceptions (with good reason), but one cool thing about them and the Haskell RTS is that you can almost [1] always cancel a thread from the outside. No need for the thread to cooperate like in Golang. The entire `async` package is built on this. The `race` combinator is an especially cool application. After doing a big project in Golang, I appreciated this more. We had our fair share of g…

Rust's futures also get this through being poll-based. You cancel a future by dropping it. Futures compose really easily, so you can combine a bunch of futures in interesting trees of selects and joins and any that are not completed get cleaned up automatically and with little/no overhead when they go out of scope / when the task they're a part of completes. You don't think about them like separate chunks of work, yo…

> As long as you don't go out of your way to write bad async code

I've seen the light of async Rust and I believe (heh, sorry for the semi-flippant sarcastic remark here but I do genuinely love it when I do things right with async Rust) but I feel that writing bad async code is also not something that the compiler will actively dissuade you from. It goes to certain lengths but not quite far enough IMO.

I didn't want to do it but at one point I started reading how is async implemented and that actually lifted a big part of the mystical veil and helped me understand it better. Now if I can also completely internalize the lifetime semantics combined with async I'd be very proud of myself. (But it doesn't help that I am not working with Rust currently.)

Re: The case of a leaky goroutine

#63

Post by pdimitar »

psnehanshu wrote: post #33

Just wondering if threads in Rust can suffer such problems? Background: I am coming from the JS/TS/Node world, and have decided to jump onto a compiled language. I narrowed down my choices to Go and Rust and eventually decided to go with Rust, because it didn't use GC for memory management.

I have several production Rust projects under my belt and while I am nowhere near the level of most full-time Rust devs (I don't work with Rust full-time, it's opportunity based for me) I have noticed that it's very difficult to introduce such leaks with Rust.

It is possible to do it by using Arc / Rc and cyclical references (which the borrow checker makes infuriatingly difficult -- for good reason!) but you really have to go out of your way for it. And while there are real projects where you need such idioms I have found that you should not try to twist Rust's arm and just opt for something like arena allocator.

Re: The case of a leaky goroutine

#64

Post by pdimitar »

subomi wrote: post #60
masklinn wrote: post #18

The explanation of the issue in ToDoneInterface really is not clear to me because of this: > The defer close() seems to close well, but it’s on the wrong channel. The `done` input channel is supposed to be closed by a caller, and the goroutine is closing the output channel, surely that's the point? Now from what I know of go channels and understand of the code involved, the `done` channel may never get closed by the…

At this point, I think done chan should be an anti-pattern. Just use context for coordination and cancellation.

I get how a Context can be used for cancellation but how is it used for coordination?

Re: The case of a leaky goroutine

#65

Post by favflam »

hedora wrote: post #4
lxe wrote: post #3

Thread leaks, which happen more frequently due to threaded async abstractions, such as the goroutine, are less often discussed than memory or CPU leaks, but are much more dangerous in multi-tenant container environments. A thread leak can lock up your entire node, including all the control plane processes. A container spec doesn't provide an easy way to control thread/nproc/ulimit limits (you can still do it, but it'…

My #1 complaint about Rust is that leaking a future is safe. It means the compiler can’t check for async coroutine leaks, and it breaks the borrow checker’s ability to say “nothing else has a reference to this any more”. Anyway, we’re using golang for some stuff at work, and holy crap, I forgot how terrible it was to work in high level languages that don’t statically check for correct synchronization. If C++-style co…

For what it is worth, I totally agree. I programmed a fairly large project only to find that goroutines were mysteriously blocking in extremely hard-to-debug subtle ways. So, I had to rewrite a huge chunk. I think goroutines are like Ackbar "It's a trap" trap.

I was liberal in usage with goroutines and channels. But after that experience, I decided to religiously track each goroutine in my head. If I reached the point of not being able to mentally map all goroutines running, I would cut out go. I started using callbacks more too to avoid the pernicious blocking I have experienced.

Re: The case of a leaky goroutine

#66

Post by silisili »

dagss wrote: post #51
bakul wrote: post #17

If Go allowed something like "handle = go foo()", goroutine could be automatically terminated when handle goes out of scope or becomes dead and is garbage collected. You can also use handle to cancel a goroutine etc. Go designers specifically avoided this model (of having a goroutine "id") for reasons I don't remember any more (may be to avoid making them heavier weight?) but this would be one way to stop leaky gorou…

The convention of passing ctx does almost the same thing though. Make a new context.WitCancel and pass it to the goroutine. It just requires programmer cooperation, but as long as you pass ctx all through the stack down and handle err on the way back, it is not often you deal with it explicitly.

It does, but feels extremely bolted on(because it was). To do properly , near every single function and method has to take a context, which gets repetitive. Then every single thing in the call stack needs to actually check for cancellation.

It feels like something similar should be baked in, and inherited from its parent by default(but overrideable). And a cancel would cancel the callstack. Would be nice to make this cleaner in go 2, I think.

Re: The case of a leaky goroutine

#67

Post by Karrot_Kream »

physicles wrote: post #59
Karrot_Kream wrote: post #43

Earlier quoted context omitted.

`fx` is mostly just Dependency Injection in Go which has been a thing in Java forever. I'm curious though, when do you reach for `fx` in a non-industrial project and when do you not? I still use the same patterns of separating out the implementation from the interface but I've been wiring in the dependencies by hand. I'm curious if folks reach for `fx` immediately or if it's something that requires thought to add. Th…

I still wire things up by hand: pass concrete parameters, use defer foo.Close(), use context.Context for signaling close. It’s super obvious how stuff works. In a 200k line code base, there are maybe 2 components where I haven’t been able to simplify the shutdown sequence to my satisfaction, but I doubt a framework would add clarity there. Maybe I just don’t know what I’m missing.

I found fx to be good for fast iteration, think quickly shipping startup code. You can just sorta spaghetti wire everything together and it's mostly okay. Though one can ask why in that situation you aren't using a dynamic language which is a good question.

Re: The case of a leaky goroutine

#68

Post by masklinn »

sapiogram wrote: post #55
Thaxll wrote: post #41

Earlier quoted context omitted.

So it's absent from every language but Rust?

Javascript is even better, by not having multithreading at all. I am not joking, Go is much worse than Java and C#, but Javascript and Rust are the only mainstream languages where I've seen non-experts reliably write correct concurrent code. Maybe it's true for langauges like Elixir as well, but I haven't tried.

My experience of JavaScript is nowhere near that good.

While it does not have parallelism, the concurrency is as unrestricted as it is in go, forgetting awaits is a very common issue leading to wild tasks running unbounded and unchecked, and when you need any other synchronisation mechanism you have to write them yourself and they’re tricky indeed.

$dayjob’s JS team keeps chasing concurrency issues, and breaking builds because of them.

Re: The case of a leaky goroutine

#69

Post by prosunpraiser »

atombender wrote: post #22

I wish Go recorded the timestamp of goroutine and let you access them. An app I work on recently had a bug where goroutines would slowly build up over time. Turns out the bug is in the Growthbook SDK [1]. We can monitor the number of goroutines, but having a large number of goroutines waiting in the location that gets stuck is normal — we can only see such a problem over multiple days, in that the minimum value slowl…

Execution traces have a goroutine profile which outputs the count of goroutines as well. That can be used for an alert as well - though it would require parsing the trace output. They recently made some changes to give a structured API over trace data - maybe use that?

Re: The case of a leaky goroutine

#70

Post by pdimitar »

vonwoodson wrote: post #50

While The White House (and actual Rust and Go enjoyers) are advocating for these safe memory-safe languages; what is really going on is that the warts these languages have are just not as well known yet. In a few years time Go will be just as hated as C++, and there'll be some new darling programming language that'll "solve all out problems". To be fair, I do look forward to when logic programming languages get their…

We can't go from X% error rate to 0% error rate, sadly, even though we want to.

But if we can go to (X/2)% error rate then that's still a win.

I wouldn't mind if we replace Golang and Rust in 10-ish years or so. For now they are definitely doing better than C++, especially having in mind that the old guard is gradually retiring and the newer generation are not as good with it.

You seem disappointed that we haven't found the one true universal language yet. I am as well, but no need to trash-talk the current iterative improvements. Apparently that's how we'll get to that ultimate thing.

Post reply on HN