Live data from Hacker News

Notes on structured concurrency, or: Go statement considered harmful

vorpus.org

101–110 of 234 posts

Re: Notes on structured concurrency, or: Go statement considered harmful

#101
post #32

Earlier quoted context omitted.

+1. This is structured programming, applied to concurrent setting. The argument for structured programming was made and won in the '60s [0]. It is amazing that we keep making the same mistakes over and over again. > The unbridled use of the go to statement has an immediate consequence that it becomes terribly hard to find a meaningful set of coordinates in which to describe the process progress. Usually, people take…

> The argument for structured programming was made and won in the '60s Most code out there is fine using break and continue in loops, and early returns are pretty popular. All of these are unstructured programming (by the 60s definition). I don't think it's all that clear that it has won.

The author anticipates this point, and rebuts it:

"In the end, modern languages are a bit less strict about this than Dijkstra's original formulation. They'll let you break out of multiple nested structures at once using constructs like break, continue, or return. But fundamentally, they're all designed around Dijkstra's idea; even these constructs that push the boundaries do so only in strictly limited ways. In particular, functions – which are the fundamental tool for wrapping up control flow inside a black box – are considered inviolate. You can't break out of one function and into another, and a return can take you out of the current function, but no further. Whatever control flow shenanigans a function gets up to internally, other functions don't have to care.

This even extends to goto itself. You'll find a few languages that still have something they call goto, like C, C#, Golang, ... but they've added heavy restrictions. At the very least, they won't let you jump out of one function body and into another. Unless you're working in assembly, the classic, unrestricted goto is gone. Dijkstra won."

I tend to agree with the author. Linux kernel code, for example, uses goto for error handling, particularly when implementing system calls. But the author is correct: the gotos can't jump out of the function. Readers can still infer control flow from function call sequences. I quite enjoyed the post, I think it's worth giving the author's claims serious consideration.

Re: Notes on structured concurrency, or: Go statement considered harmful

#103
post #22

I LOVE Go's concurrency model, but this article has won me over (pending some experimentation anyway). If you just skimmed, this is actually worth a careful read. The parallels between "go" and "goto" are explained very clearly, and you get some awesome Dijkstra quotes to boot!

I think people are missing the merits of what the author said in their defensiveness of their existing approaches, which is a bit sad.

The article had a whole section on how any challenges to existing paradigms and tools will be met with fierce opposition. The comments here are an excellent illustration!

Re: Notes on structured concurrency, or: Go statement considered harmful

#104
post #2

I thought the title was kinda clickbaity, but it turned out to be a great article. Also the comparison to goto really effectively conveyed the point he was trying to make. I have two questions though: * Does anything else like this currently exist (other than the Trio library he mentions), which shows that it's a superior paradigm in practice? * What are the cons to this approach? Why not do it?

I've written a similar thing in C (there's a link in the article), here's the intro: http://libdill.org/structured-concurrency.html

Re: Notes on structured concurrency, or: Go statement considered harmful

#105
post #58

Earlier quoted context omitted.

The built in thread pools must have a limit. Also, this isn't just a matter of keeping thread resources down, it's supporting scenarios like developing web and UI software. For example, if I have "OnButtonClick" and it does async work, I'd like to also call other functions that may also touch UI components. WPF has support for binding async actions to UI events, and it eliminates the need to ever wonder what thread y…

> The built in thread pools must have a limit. The go runtime has been shown to handle millions of goroutines, and there are performant and scalable programs, like Cloudflare's RRDNS that run tens of thousands of goroutines ( https://blog.cloudflare.com/quick-and-dirty-annotations-for-... ). There's plenty of web software using this model written in Go. I haven't done UI programming in Go - but in your example, I'd i…

> The go runtime has been shown to handle millions of goroutines,

Fair enough.

> I haven't done UI programming in Go - but in your example, I'd imagine you could simply have an event launch a goroutine...

But now you are in the "goto" mess that this article describes.

Async/await allows me to to write "goto safe" async code synchronously, for UIs.

Re: Notes on structured concurrency, or: Go statement considered harmful

#106

A point I believe the author has missed: Goroutines aren't simply forked functions, but are abstracted as independent, autonomous processes. Now, that doesn't mean 'go' is a sufficient tool to reason about goroutines, but this may also not be as useful a solution as he touts it to be.

I don't think that's really relevant to his point. His argument is about control flow, rather than capabilities. The library he has built is designed to provide any functionality that didn't previously exist, but rather to make code that accomplishes the same tasks as before easier to reason about and somewhat safer to write.

An implementation of this pattern could handle the spawning of functions however it wants and the language supports (as independent processes, threads, with an event loop, etc.).

Re: Notes on structured concurrency, or: Go statement considered harmful

#107
post #20

Earlier quoted context omitted.

I suspect that if you were to try to do this in real code, something like an onclick handler would have to be run in a nursery scoped at the page level, by the code running the page. An onclick handler on its own accord can't run itself in the correct nursery. It's like the network handling case there, where the nursery's lifetime is either unbounded, or tied to the lifetime of the OS process, depending on how you wa…

Maybe a subtle difference here is that a page doesn't get to block on anything before it gets closed. So how is creating a nursery that matches the lifetime and visibility of the page different from not using a nursery at all?

"Maybe a subtle difference here is that a page doesn't get to block on anything before it gets closed."

In real browsers, the UI act of closing a tab and the completion of cleaning up all of its resources are already clearly separated. I can see that when I terminate a browser with a lot of tabs and the main window has entirely closed while the browser runs at 100% CPU for quite a few more seconds. Plus a browser can probably hard-kill running Javascript code with some reasonable effectiveness. (Not sure. Asynchronous exceptions are very, very hard.)

So I don't think that's a disqualifier.

"So how is creating a nursery that matches the lifetime and visibility of the page different from not using a nursery at all?"

Well, as I've said in one of the reddit conversations, bear in mind the entire purpose here is not to enable something that was previously impossible, but to constrain us from using primitives in their most powerful form. The big thing is that if you had a nursery-based system, and the page had a nursery associated with it, and the page's render routine returned, you'd know that all the page threads must necessarily be terminated. You can't know that with the same confidence now, because the programming style does not permit that level of confidence by construction.

Browsers are a bit of a pathological case on a lot of levels, though, and probably not a great mental example, because you can assume a high degree of competence, concern, and skill in the people programming the browser, and they do things like use static analysis all the time and even in the "worst" cases, develop entire new programming languages to write browsers in. So you are probably justified in saying "But jerf, I'm pretty confident the browsers are already cleaning up their stuff without this stuff." The real question is, how does this look for Joe Programmer and his ability to work in the domain of multithreaded programming, which is well known and widely acknowledged to be very difficult, by constraining what mistakes he can make?

One of the other angles you can look at this with is, if I have two pieces of correct code and I compose them together, are they still correct? The nursery system says that with my nursery, I can call other code that uses them, and the composition attained over that function call is also correct and does not leak resources. We do not get that guarantee with some other primitives. We do with others. There's a lot of experimentation still going on in this field. I'm not saying this approach is guaranteed to be correct, but it's one of the more plausible claims to being a primitive as basic as "if" is relative to "goto". Compare with Software Transactional Memory, which as nice as it may be, is wildly more complicated than "if" no matter how you slice it.

Re: Notes on structured concurrency, or: Go statement considered harmful

#108

Earlier quoted context omitted.

“Formal verification” for Rust code -- does that actually exist yet, or do people just hope/assume that the language will be proven consistent and that awesome theorem-checking tools will emerge eventually? I agree that Rust has a great model that seems to lead to very solid code, but “formal verification” is a high bar to clear.

Some people are actively working on formal verification for Rust. The RustBelt project is the first that jumps to mind. A paper and some discussion here: https://news.ycombinator.com/item?id=16302530

[deleted]

Re: Notes on structured concurrency, or: Go statement considered harmful

#109
post #59

Earlier quoted context omitted.

It's not only that Rust allows formal verification, it's that it does it by default, writing unsafe Rust code for no serious reason is fundamentally frowned upon, and even then there is a serious effort to bring some tooling to bring more confidence even on "unsafe" Rust code. > But I suspect the truth is that "njs was right", and I hope the it's sooner rather than later. I'm not sure. The problem can and has been so…

“Formal verification” for Rust code -- does that actually exist yet, or do people just hope/assume that the language will be proven consistent and that awesome theorem-checking tools will emerge eventually? I agree that Rust has a great model that seems to lead to very solid code, but “formal verification” is a high bar to clear.

We have a dedicated WG now actively working on it https://internals.rust-lang.org/t/announcing-the-formal-veri...

Re: Notes on structured concurrency, or: Go statement considered harmful

#110

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

You realise that makes you sound exactly like an old Fortran programmer refusing to give up goto? A for loop is just a wrapped goto! ;-) I think the author proposes a useful way to structure multi-threading. The comparison to goto isn't perfect and he needs to play fast and loose with some terms to keep the analogy working but he makes a good point. I'm not convinced yet that the nursery pattern should be the only al…

"I think x is like goto."

"I disagree; I don't think x is like goto."

"Now that we've agreed that x is like goto, not wanting to give up x makes you like a big dinosauric dummy~"

Post reply on HN