Live data from Hacker News

Notes on structured concurrency, or: Go statement considered harmful

vorpus.org

131–140 of 234 posts

Re: Notes on structured concurrency, or: Go statement considered harmful

#131

Earlier quoted context omitted.

Rust has co-routines on nightly ("generators"), and it's an important underlying aspect of async/await.

When I said coroutines, I really meant M:N thread scheduling. I could see how that can be confusing.

Yes, that's very different :)

That's Tokio in Rust.

Re: Notes on structured concurrency, or: Go statement considered harmful

#132

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

It really is a whole wrapped up group of existing constructs, but that doesn't mean there isn't merit in it. The author mentions how this was central to Dijkstra's proposal for structured programming: "And now that Dijkstra understood the problem, he was able to solve it. Here's his revolutionary proposal: we should stop thinking of if/loops/function calls as shorthands for goto, but rather as fundamental primitives…

> but that doesn't mean there isn't merit in it

Of course not. It also isn't original, or a "silver bullet" that I find relevant to general concurrent programming.

> I don't really understand what you mean by this.

I was trying to keep my rant a bit short, but my point is that the only practical example for a post that puts a lot of effort into complaining about a core Go construct (it's the title) does not even remotely apply to Go.

This is partly due to Go being shaped around this very core construct.

> Why exactly is branching better?

"goto" can (sometimes, as there are many valid usecases) be problematic as it can make control flow very obscure when read. "go" is extremely clear to read, and only has the concern that you don't know if a given function has created goroutines. It is of course usually described by documentation, or implicit from functionality that such thing will occur, and unlike goto, the code is extremely clear about what is going on if you read it.

Furthermore, whether a function call has created goroutines is by itself not a concern. What can be a concern would be if some types of resources that must be closed (i.e. a file) is referenced after you close it. That is not related to concurrency, but lifetimes.

By lifetimes, I do not necessarily mean Rust-style compiler-enforced lifetimes (which I like), but simply API contracts. A library may store a reference to something you pass, and may use this reference in later calls, potentially after you invalidated the reference by closing a file descriptor.

A joined execution does not even remotely solve this problem, as it is not related to concurrency. It solves a different problem, related to just managing concurrent execution.

> These are all related concepts. They are about doing different things "at the same time, ...

They are not at all. Callbacks are often used together with certain types of concurrency constructs, such as an event-loop that can call callbacks upon various events. In JS, for example, the concurrency construct is a single global event-loop that calls tasks/microtasks.

Callbacks themselves, however, are not a concurrency construct. Thus, comparing them to concurrency constructs is very weird.

(A SAX parser is an example of a non-concurrent use of callbacks.)

> Even using threading with a GIL can create execution paths that you aren't really aware of until you actually look through the code or debug.

It does not create execution paths that are not easily visible (you know nothing about a program until you look through the code), but yes, threading.Thread can lead to some unanticipated behavior.

Note that I was excluding this as a parallelism construct, not a concurrency construct. The behavior will be some form of cooperative multi-tasking.

EDIT: too much "however".

Re: Notes on structured concurrency, or: Go statement considered harmful

#133
post #14
post #11

How is this different than calling join(), after spawn()?

How is if different that just using goto to go to either the true or false clause? It isn't about what you can do, it's about what you can't do if you use this mechanism (this restricts where you can call spawn and join) and the guarantees you can then build on top of that.

Given the "escape hatch", there is no difference to passing a thread pool object around (or referencing it otherwise), that would for ex join on destruction.

And this is a valid approach. But pretending that this should be the only one and that this is in a way similar to unstructured goto vs structured programming? I'm not buying it. Because there will be long lived global nurseries floating around in big enough codebases, effectively eliminating all the guarantees they are supposed to provide for the affected threads. I mean; I'm not sure they can even guarantee the advantages they are supposed to provide (in the sense of providing new easy to check properties, with actual tools existing capable of checking them).

Don't get me wrong. I find the approach interesting, and will happily use it where applicable, but just the comparison to goto does not really makes sense, nor does the fiction that threads are best modeled by always being contained into managing function calls (hmf, except when they are not...). The "escape hatch" is so big that it just plain devalues the solution compared to not having it (or having it only in vastly more constrained ways) and then obviously not pretending this is what should replace traditional spawning (and even more) everywhere.

Re: Notes on structured concurrency, or: Go statement considered harmful

#135

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

> The "with" example for benefits to not having a "go" statement seem rather bogus, especially seeing that such RAII constructs do not exist in Go (no destructors, remember?). You've greatly underestimated how general this problem is. First, Python's `with` statement has nothing to do with destructors. From the PEPM for `with`: with VAR = EXPR: BLOCK which roughly translates into this: VAR = EXPR VAR.__enter__() try:…

I mentioned RAII because the text around "with" mentions its use with RAII.

Go does not have exceptions (panic is not meant as "normal" flow control), and therefore has no use for a "with" construct. "defer" is used for a somewhat similar purpose. Go does not have destructors, and therefore has no possible implementation of RAII.

However, none of this applies to goroutines. A goroutine only gives errors if the author decides that such is necessary. If so, it will likely be through an error channel. There is no unexpected code paths through such readout, rendering "with" and RAII useless.

So again, for a post that was very focused on complaining directly about the "go" keyword, I was expecting something applicable to Go.

Re: Notes on structured concurrency, or: Go statement considered harmful

#136

Earlier quoted context omitted.

Knowing whether a background task has spawned is very different from not being able to follow the control flow (goto potentially jumping to an entirely different function body). Now, while Go is designed mostly for you to not care about goroutines, there are some corner cases where one must know if a resource is used by anything, such as the chase of when you wish to close a file handle. However, I'd argue that this…

But this concept does also solve that problem. Maybe more simply. Rust is renowned for being a total headache with borrowing and lifetimes. Nurseries might be a simpler solution for this. As you say, a corner case. But a common one...

It actually does not solve the problem of lifetimes. It only solves a very isolated instance of it.

For example, if we take the problem of an os.File, some library might store the reference, and use it unexpectedly in a later function call after you called Close. This presents the exact same issue as a goroutine holding it.

(I honestly have no other examples of this issue than files/connections being closed prematurely for Go.)

Re: Notes on structured concurrency, or: Go statement considered harmful

#137
post #3
post #2

I thought the title was kinda clickbaity, but it turned out to be a great article. Also the comparison to goto really effectively conveyed the point he was trying to make. I have two questions though: * Does anything else like this currently exist (other than the Trio library he mentions), which shows that it's a superior paradigm in practice? * What are the cons to this approach? Why not do it?

I don't think its a superior paradigm, just a different one. I only see his nursery as being useful when you really want your async tasks to complete before the function in which they were dispatched returns. That's far from covering every use case of concurrency! A lot of the value of concurrency is in background operations. These simply can't be tied to the duration of a function call on the dispatch thread. Doing…

>I only see his nursery as being useful when you really want your async tasks to complete before the function in which they were dispatched returns. That's far from covering every use case of concurrency!

Perhaps, but only having a complex solution that handles 100% of use cases is less desirable than having a simple one that handles 80% of the most common use cases PLUS the ability to go deeper and use the complex method (if, and only if, it's necessary...)

Re: Notes on structured concurrency, or: Go statement considered harmful

#138
Interesting ideas, and well-worded.

You could achieve something similar in JavaScript with Promise.all() and await:

  await Promise.all([
    asyncFunc1(),
    asyncFunc2(),
    asyncFunc3()
  ])

Of course, that's not language-level and the point seemed to focus more on eliminating traditional branching than just adding another way to do it.

Re: Notes on structured concurrency, or: Go statement considered harmful

#139

Earlier quoted context omitted.

I think the article's point is that with Future's you can still pretty easily invoke a Future-returning function and forget to return its value, ending up with what you might call an orphan continuation.

If the future has a side effect I'm not concerned with, like sending a mail, I can't see the problem?

The problem is much like the author said -- it's easy to have errors disappear into the ether in a way that is much less likely in synchronous logic. Also, if those side-effects matter, it's easy to make faulty assumptions about time ordering.

The most obvious situation to me is in the way asynchrony exists in front-end programming and how this affects testability. If you can't actually know when a process (like an animation) ends, you can't accurately test.

In general, my experience has been that reification of abstract things often presents benefits in the long run. Reification of functions admits a whole host of techniques. Reification of classes facilitates metaprogramming. Reification of in-flight processes as promises helps with being able to compose and abstract over them. Nurseries seem like reficiation of an finite execution context.

Re: Notes on structured concurrency, or: Go statement considered harmful

#140
post #138

Interesting ideas, and well-worded. You could achieve something similar in JavaScript with Promise.all() and await: await Promise.all([ asyncFunc1(), asyncFunc2(), asyncFunc3() ]) Of course, that's not language-level and the point seemed to focus more on eliminating traditional branching than just adding another way to do it.

I had a similar thought, initially, which I posted elsewhere in this submission: https://news.ycombinator.com/item?id=16924407
Post reply on HN