Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on structured concurrency, or: Go statement considered harmful

vorpus.org

Post reply on HN
Print view View original

181–190 of 234 posts

Re: Notes on structured concurrency, or: Go statement considered harmful

#181

Post by cryptonector »

arghwhat wrote: post #19

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

Callbacks have plenty to do with concurrency: first you compile everything into CPS (where callbacks are continuations) and then a trivial event loop nets you cooperative multitasking. When you don't have language support for this but you need concurrency, you just write something resembling CPS by hand and call it "callback hell".

> (pthread_create—threading.Thread doesn't count as parallism due to GIL)

Eh? Why would you have a GIL? Maybe in Go, but so what, just don't use such a language/run-time.

Re: Notes on structured concurrency, or: Go statement considered harmful

#182

Post by atombender »

lclarkmichalek wrote: post #6

This is usually why I end up using https://godoc.org/golang.org/x/sync/errgroup instead of straight go statements, as it addresses some of the cancellation and error propogation issues. When I think of my use of naked go statements, it's usually for periodic tasks; having something similarly structured for them would be a clear win to me (though potentially the impact is less significant, as it's less painful to writ…

ErrGroup is nice, but it was created before contexts existed, and doesn't have support for cancellation. I have a bounded worker pool executor that handles cancellation that I'm currently extracting from a private project; shout out if interested.

Re: Notes on structured concurrency, or: Go statement considered harmful

#183

Post by _Codemonkeyism »

acjohnson55 wrote: post #139
_Codemonkeyism wrote: post #121

Earlier quoted context omitted.

If the future has a side effect I'm not concerned with, like sending a mail, I can't see the problem?

The problem is much like the author said -- it's easy to have errors disappear into the ether in a way that is much less likely in synchronous logic. Also, if those side-effects matter, it's easy to make faulty assumptions about time ordering. The most obvious situation to me is in the way asynchrony exists in front-end programming and how this affects testability. If you can't actually know when a process (like an a…

I'm not following, how could an error with Future[Either[A,B]] disappear compared to synchronous logic of Either[A,B]? Our code base is the same for sync and async logic and error handling.

One thing that doesn't work is Anders Hejlsbergs method of letting unchecked exceptions bubble up, but exceptions haven't been a good idea for business code anyway.

Re: Notes on structured concurrency, or: Go statement considered harmful

#184

Post by wybiral »

stouset wrote: post #178
wybiral wrote: post #126

Earlier quoted context omitted.

Most articles riding on the "x considered harmful" meme are clickbait without much content.

Nothing could describe this post (or anything on the author’s site in general) less accurately than your comment.

How so?

Re: Notes on structured concurrency, or: Go statement considered harmful

#185

Post by wokwokwok »

joshuamorton wrote: post #169
wokwokwok wrote: post #166

Earlier quoted context omitted.

The point of the construct is not explicitly waiting for child tasks/threads/whatever, but that as a formal restriction , you cannot leave the scoped block with unresolved tasks. I don't believe either of the constructs you've mentioned do this, and I'm not aware of any that do. The trivial counter example would be a deeply nested `setTimeout(..., 5000)` inside the javascript code. It doesn't matter if you've called…

>The trivial counter example would be a deeply nested `setTimeout(..., 5000)` inside the javascript code. Well of course, setTimeout isn't async/await based, its callback based. If instead you only had this api, you could make that formal assertion: await promiseSetTimeout(time).then(...); And in fact you can write promiseSetTimeout today! So basically, async/await provides this as long as you only use async/await. I…

> as long as you only use async/await...

Well, what if you don't?

That's the point.

The proposed construct does not require that: I'm not saying its better or worse; I'm just pointing out that it's different.

Obviously if you choose to avoid branching code in your functions, they won't branch; what's novel here is that the closing scope (ie. in this case, the collapse of the python with block) triggers a collection of all the ambient tasks started in that context.

I'm sure you could implement something similar in javascript, but it is not the same as Promise.all.

Re: Notes on structured concurrency, or: Go statement considered harmful

#186

Post by eridius »

stouset wrote: post #176
lilyball wrote: post #84

Earlier quoted context omitted.

Nurseries and lifetimes are orthogonal concepts. In fact, Rust already has a "nursery", except it's generally called a scoped threadpool. They use lifetimes to ensure the threads can reference stuff on the stack without copying it. So a "nursery" is basically just a scoped threadpool with a 'static lifetime.

I don’t think there’s any reason a nursery has to have a static lifetime.

The description of the nursery is you can pass it around wherever you like. More generally, this concept seems to have been designed in a language without lifetimes. You obviously can have a scoped threadpool without a 'static lifetime, but if you want to pass it around to arbitrary locations then you do.

Re: Notes on structured concurrency, or: Go statement considered harmful

#187

Post by joshuamorton »

wokwokwok wrote: post #185
joshuamorton wrote: post #169

Earlier quoted context omitted.

>The trivial counter example would be a deeply nested `setTimeout(..., 5000)` inside the javascript code. Well of course, setTimeout isn't async/await based, its callback based. If instead you only had this api, you could make that formal assertion: await promiseSetTimeout(time).then(...); And in fact you can write promiseSetTimeout today! So basically, async/await provides this as long as you only use async/await. I…

> as long as you only use async/await... Well, what if you don't? That's the point. The proposed construct does not require that : I'm not saying its better or worse; I'm just pointing out that it's different. Obviously if you choose to avoid branching code in your functions, they won't branch; what's novel here is that the closing scope (ie. in this case, the collapse of the python with block) triggers a collection…

>Well, what if you don't?

This is kind of a silly question. Its always possible to subvert a safe construct system if you try hard enough. You can write unsafe blocks in rust. You can pass in a callback to a nursery, as is described in the article.

>The proposed construct does not require that

Well, kind of. The article actually explicitly states that

>Here's a simpler primitive that would also satisfy our flow control diagram above. It takes a list of thunks, and runs them all concurrently:

which is equivalent to promise.all, satisfies the invariant nurseries do, except in very specific circumstances (unbounded while loop-y constructs). And you just simply can't use promise.all in that situation.

Re: Notes on structured concurrency, or: Go statement considered harmful

#188

Post by mattnewport »

pmarreck wrote: post #134

I don't like the term "nursery" (maybe "highway" or "complex" or... something else) but this seems to be a good design change, unless I'm missing something

Yeah, I thought the article was interesting but the choice of "nursery" still has me scratching my head. I can't see where it comes from or why that word was chosen.

Re: Notes on structured concurrency, or: Go statement considered harmful

#189

Post by wokwokwok »

joshuamorton wrote: post #187
wokwokwok wrote: post #185

Earlier quoted context omitted.

> as long as you only use async/await... Well, what if you don't? That's the point. The proposed construct does not require that : I'm not saying its better or worse; I'm just pointing out that it's different. Obviously if you choose to avoid branching code in your functions, they won't branch; what's novel here is that the closing scope (ie. in this case, the collapse of the python with block) triggers a collection…

>Well, what if you don't? This is kind of a silly question. Its always possible to subvert a safe construct system if you try hard enough. You can write unsafe blocks in rust. You can pass in a callback to a nursery, as is described in the article. >The proposed construct does not require that Well, kind of. The article actually explicitly states that >Here's a simpler primitive that would also satisfy our flow contr…

Oh come on.

I don't agree that the proposed idea is fundamentally new and amazing; but I think it is novel, and there may be some value in being able to semantically bind tasks to execution points, specifically when the tasks are spawned in naive (or uncontrolled, eg. library) code, and have not explicitly opted in to the scheme.

Does this particular implementation do that perfectly? No, probably not.

...but I think the idea of it has some merit.

There's more value here than just waiting for a series of deferred tasks that you have explicitly created, and explicitly opted into a specific concurrency workflow with.

Note specifically these parts of the proposal:

> we declare that a parent task cannot start any child tasks unless it first creates a place for the children to live

^-- This is not satisfied by your method, at all.

> But the problem with this is that you have to know up front the complete list of tasks you're going to run, which isn't always true.

Literally the next sentence after the one you quoted.

> What if you really do need to write a function that spawns a background task, where the background task outlives the function itself? Easy: pass the function a nursery object. There's no rule that only the code directly inside the

Notice how you can avoid the 'setTimeout(() => { .. })` in nested code issue by explicitly passing in a context that has its own scope.

...

There's a lot of stuff in here that is actually quite thoughtful.

You don't have to dig through it, finding every little nitpick you can to call it out on; just think about the idea being proposed.

It's certainly not 'just promises'; if you think it is, then... I don't know what to say. You're wrong. :P

Re: Notes on structured concurrency, or: Go statement considered harmful

#190

Post by _Codemonkeyism »

z0r wrote: post #96
acjohnson55 wrote: post #91

Earlier quoted context omitted.

I think the article's point is that with Future's you can still pretty easily invoke a Future-returning function and forget to return its value, ending up with what you might call an orphan continuation.

Wouldn't the unreferenced Future, once fulfilled, be garbage collected?

Yes
Post reply on HN