What Color Is Your Function? (2015)

Alright. One reason for my confusion is that I assumed you would post "async functions (and generators?) are not colored functions in JavaScript" at the top level if you meant that, since the article is about async functions in JavaScript. So I wondered, what's the distinction between async functions in JavaScript and in Python that makes one "colored functions" but not the other? But this is not what you intended. Thanks for the clarification.

Some programming languages have stackless async functions and generators that work the way you said, and some other languages have stackful coroutines. In the languages I use the most, functions that switch to other parts of the program are functions (according to the type system and the calling convention, but not in the sense that they have only one entry point and one exit point and no state). This is pretty great, because things are composable in precisely the way the article complains that they are not. Given that these things exist, it's fine to say "coroutines and functions are two different things" but maybe saying "a function which jumps to a different stack and which can then be jumped back into is not a function" will just make people very confused about the whole thing.

If you have this, uh, code fragment:

  function read_five(socket)
    local data, err = socket:recv(5)
    if not err then
      return data
    end
    return nil, err
  end

perhaps we shouldn't say "the value declared by the code fragment is a function or is not a function depending on whether recv nonlocally switches only to the kernel or whether it can also switch to an event loop in the same process"

Go does not have exception (exactly because of this problem.)

For passing information, you use channels, which can pass more than one value back to the caller.

The big thing is not running tasks in the background but the tight integration of channels and runtime scheduler that allows having an invisible event loop on top of what is synchronous programming.

Sometimes you want to block the main thread, or at least finish what you were doing. With async/await and cooperative concurrency you can be explicit about what will run on a thread. You retain control of the thread until you yield or await. You can ask tasks to complete on other threads or post back to the main thread. You have a lot of control. Its easy to write code without locks that runs concurrently on the main thread but is still able to build a UI in a threaded way.

I don't really know how go UI frameworks work. How do you have multiple, preemptively scheduled goroutines on the UI thread but without critical sections? You have to use channels and message passing back to a main thread manager to handle this, yes?

I think Java's Loom would have the same issue but again, I don't really know. Perhaps worrying about a UI thread is 'fighting the last war' and we should work on new UI paradigms in these new language features.

Interesting read.

I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge).

In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures.

What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer.

I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them).

I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.)

Here is one example of where it is used: https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db...

In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity).

I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.

Right. Even POSIX.1 has the same "messaging" functionality, so:

* spawn/1 becomes fork()

* send/2 becomes kill()

* receive F -> ... G -> ... end becomes sigaddset(&s,F);sigaddset(&s,G);sigwait(&s,&r)

There are of course other issues, like there aren't that many signals, or that memory management is hard, or whatever, but these are solved by other languages as well, so you can imagine this is at least straightforward to implement.

The downside is that it means you can have functions that you can't write. That can be a big bummer to a lisp programmer, but I don't think python, javascript, etc, programmers care about that, so I think history will judge the trend to colour functions as lazy and shortsighted.