Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on structured concurrency, or: Go statement considered harmful

vorpus.org

Post reply on HN
Print view View original

141–150 of 234 posts

Re: Notes on structured concurrency, or: Go statement considered harmful

#141

Post by coldtea »

lilyball wrote: post #86
pnathan wrote: post #76

Article persuasive prima facie and arguments plausible. I've had to reinvent a structured method of managing threads a number of times, unfortunately. Author is a PhD student, which bodes well for not reinventing wheels dumbly. Therefore, I look forward to the lit review of other concurrency & parallelism work through the last 40 years, which this writeup notably lacks (author mentions his stack of papers to review).…

> Author is a PhD student, which bodes well for not reinventing wheels dumbly. Except that's exactly what the author did. They just reinvented scoped threadpools.

In the sense that a car is a reinvention of a horse buggy.

Re: Notes on structured concurrency, or: Go statement considered harmful

#143

Post by arghwhat »

tomtheelder wrote: post #118
zamalek wrote: post #99

Earlier quoted context omitted.

Agree with everything. I think the misunderstanding here is that goroutines are not classical threads (which would share many problems with `goto`) - they exist somewhere between coroutines and actors (because of channels+select). Actors are an established and mature solution to many headaches surrounding concurrency and parallelism. Goroutines share more in common with method calls (which are a form of branching) an…

I don't think that's a misunderstanding at all. The argument is all about control flow and programmer understanding. The actual underlying mechanisms aren't what he is arguing against, instead he dislikes the potential for programmer confusion when code is being executed that wasn't expected. That is 100% possible with goroutines. Method calls aren't really branching. The code path is still totally linear. You could…

> That is 100% possible with goroutines.

It really isn't. The big difference is that "goto" can lead to code you read being grossly misunderstood due to complicated flows, potentially even absurd things like jumping to a different function. (And as you mention, good code can be written with goto's—it can make particularly error handlers much easier to read in C.)

This is not the case at all with "go", which is extremely clear as to what it does and how the flow will go.

However, as a caller, you do not know the flow of the function you call unless you read it. As with any type of asynchronous programming, a function might have scheduled something for later execution: It might be a future, a promise, a timeout/interval, network receive callbacks, or a goroutine.

That is, strictly speaking, none of your concern as long as the function lives up to its contract. If not, all bets are off regardless.

> ... it just requires a lot of added thought, and the potential for mistakes is much higher.

I get your point, but I must disagree that asynchronous programming (which is what goroutines is merely an implementation of) increase the complexity of writing good code, nor increase the likelyhood of state ownership mismanagement in any measurable fashion.

> The author is not proposing any functionality that doesn't already exist, just a new control pattern to reduce the chance of creating problems.

I don't really find that the author suggests anything new at all. Rather, the author takes an existing blob of code and claims a new benefit from it.

EDIT: erroneous "in which case" replaced with "if not", as initially intended.

Re: Notes on structured concurrency, or: Go statement considered harmful

#144

Post by Osiris »

_bxg1 wrote: post #138

Interesting ideas, and well-worded. You could achieve something similar in JavaScript with Promise.all() and await: await Promise.all([ asyncFunc1(), asyncFunc2(), asyncFunc3() ]) Of course, that's not language-level and the point seemed to focus more on eliminating traditional branching than just adding another way to do it.

That's exactly what I was thinking. Some of us are probably are using this type of control flow. For me a common scenario is something like

    const promises = files.map(readFileAsync); // "nursery"
    const fileContents = await Promise.all(promises); // "with"
I generally agree with his premise that it sucks having to figure out if a function is concurrent or not; i.e., does it return a value or a Promise/Future. I'm not sure if his solution solves that particular issue though, unless it's handled automatically in his "nursery.start_soon" function.

Re: Notes on structured concurrency, or: Go statement considered harmful

#145

Post by Osiris »

pmarreck wrote: post #134

I don't like the term "nursery" (maybe "highway" or "complex" or... something else) but this seems to be a good design change, unless I'm missing something

I believe it's used because it keeps track of "children", or child functions spawned by the current function. Without knowing that background however, it's not immediately obviously to someone that hasn't heard the term what it means.

As others have mentioned, reusing the "await" keyword could cover a lot of these nursery scenarios.

Re: Notes on structured concurrency, or: Go statement considered harmful

#146

Post by Osiris »

> whenever you call a function, it might or might not spawn some background task. The function seemed to return, but is it still running in the background? There's no way to know without reading all its source code, transitively. When will it finish? Hard to say.

This reminds of me "colored functions" (red vs blue) where it becomes imperative to know if a function you are calling returns a value or a Future/Promise.

Some languages allow annotating a function to indicate as such so the IDE can help. His particular solution he presents actually doesn't address this question: Is your function sync or async? You still have to know when calling a function if it's async and needs to be in a nursery or not.

Should a programming language abstract away whether a function is async or not? async/await is a step forward (C#/JS) but it still requires knowing if the child function is async or not.

Re: Notes on structured concurrency, or: Go statement considered harmful

#147

Post by kllrnohj »

rumcajz wrote: post #123
Animats wrote: post #113

This addresses the wrong problem. The real issue is control over data shared between threads, not control flow. C/POSIX type threads have no language support for indicating what data is shared and which locks protect which data. That's a common cause of trouble. The big question in shared memory concurrency is "who locks what". Most of the bugs in concurrent programs come from ambiguities over that question. Early at…

Almost all attempts at CSP-style programming in the end resorted to sharing data to get a little bit better performance. I wonder whether we shouldn't have used a bit of speedup that Moore's law gave us to cover that cost and be done with all the shared state headaches.

Except shared data doesn't give you a little bit better performance, it gives you massively better performance. Or, in some cases, it's the only way to get usable performance at all.

Now what you could do is break objects down into annotated types. Consider immutable vs. mutable in combination with thread-unsafe, thread-compatible, and thread-safe. Immutable data that's not thread-unsafe you can share freely across threads, all is well. L2/L3 caches are happy. Mutable that's thread-safe can similarly be shared at will. Then you can force that thread-compatible objects be wrapped & accessed only from a Mutex or transfered between threads as part of a move operation.

Rust gives you the tools to do all of this, and indeed does some of it, but as part of the steep learning curve of the ownership model.

Re: Notes on structured concurrency, or: Go statement considered harmful

#148

Post by catnaroek »

Lately, I am of the idea that the real problem with how we do concurrency is that we have yet to figure out a way to do it without first-class procedures. When we spawn a thread, even in a low language such as C, we use something to the effect of:

    launch_thread(function, perhaps, some, initial, data);
The trouble with this approach to concurrency is twofold:

(0) It forces a hierarchical structure where one continuation of the branching point is deemed the “parent” and the others are deemed the “children”. In particular, if the forking procedure was called by another, only the “parent” continuation may return to the caller. This is unnatural and unnecessarily limiting. Even if you have valid reasons to guarantee that only one continuation will yield control back to the caller (e.g., to enforce linear usage of the caller's resources), the responsibility to yield back to the caller is in itself as a resource like any other, whose usage can be “negotiated” between the continuations.

(1) It brings the complication of first-class procedures when it is often not needed. From a low-level, operational point of view, all you need is the ability to jump to two (or more) places at once, i.e., a multigoto. There is no reason to require each continuation to have a separate lexical scope, which, in my example above, one has to work around by passing “perhaps some local data” to `launch_Thread`. There is also no reason to make “children” continuations first-class objects. If you need to pass around the procedure used to launch a thread between very remote parts of your program, chances are your program's design is completely broken anyway. These things distract the programmer from the central problem in concurrent programming, namely, how to coordinate resource usage by continuations.

Re: Notes on structured concurrency, or: Go statement considered harmful

#149

Post by kybernetikos »

platinumrad wrote: post #110
speleding wrote: post #89

Earlier quoted context omitted.

You realise that makes you sound exactly like an old Fortran programmer refusing to give up goto? A for loop is just a wrapped goto! ;-) I think the author proposes a useful way to structure multi-threading. The comparison to goto isn't perfect and he needs to play fast and loose with some terms to keep the analogy working but he makes a good point. I'm not convinced yet that the nursery pattern should be the only al…

"I think x is like goto." "I disagree; I don't think x is like goto." "Now that we've agreed that x is like goto, not wanting to give up x makes you like a big dinosauric dummy~"

> 'A for loop is just a wrapped goto!'

corresponds fairly closely with

> a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc.

The fact that the 'nursery' can and has been implemented with more 'primitive' constructs is actually a point of similarity with goto/structured programming not a point of difference. The GP didn't seem to spot this, despite making it their first complaint, so it was good that the post you're responding to did.

The post you're responding to is not calling the GP a dummy because of a disagreement, it's pointing out the apparent criticism the GP levelled was exactly the same criticism levelled against structured programming, and ultimately goto lost.

Re: Notes on structured concurrency, or: Go statement considered harmful

#150

Post by arghwhat »

speleding wrote: post #89
arghwhat wrote: post #19

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

You realise that makes you sound exactly like an old Fortran programmer refusing to give up goto? A for loop is just a wrapped goto! ;-) I think the author proposes a useful way to structure multi-threading. The comparison to goto isn't perfect and he needs to play fast and loose with some terms to keep the analogy working but he makes a good point. I'm not convinced yet that the nursery pattern should be the only al…

As someone whose work includes hardware driver programming, I'd rather not give up goto. It would make a lot of error handling extremely cumbersome, and much less readable. :(

However, I find "go" and "goto" to not intersect at all. I've written this many times in other comments on this thread, so I'd rather not type it out again, but the TL;DR: is that "goto" can make understanding a function when read difficult, while "go" is clear when read. No function is understood if never read, and spawning "background tasks" is a core part of asynchronous programming (and thus not an unusual side-effect).

The "nursery pattern" is a decent construct that I have used quite often whenever I felt a need, but it doesn't appear to really solve any issues mentioned in the post. I also elaborated on this quite a few times already, so TL;DR: the only real problem of goroutines is things like references to potentially closed objects, but any method may end up storing an internal reference used at a later call, making the issue not related to concurrency.

Post reply on HN