Asynchronous programming and cooperative multitasking

What y'all need is concurrent ML. I don't know why, but it still seems rather unknown. For me it hits the sweet spot of power and flexibility (generalising the actor model and still not having to do the nitty gritty details of reagents). Guile scheme has what is probably the nicest implementation (utilizing multiple cores) and the best introduction:

https://github.com/wingo/fibers/wiki/Manual

They're a step towards it, but they don't really solve the callback after callback on their own. You need something like a ->get() or await on them to be able to serialize the flow of things to completely solve that.

But that encapsulation of success or failure does do wonders for helping the swallowing of exceptions, by basically wrapping the async operation in a try/catch that carries it over for you.

Since Java doesn't have first class await support, I'm super excited about Project Loom. Imo, using green threads also has one huge benefit over callbacks: legacy code, small hacks, PoCs written in a simple and imperative manner should be much easier to port to it.

Good job at highlighting async as being cooperative multi-tasking. Those of us old enough will remember the fanfare when both Mac and Windows moved from cooperative to pre-emptive multi-tasking models. To adapt a well known phrase, every non-trivial cooperative application includes a half-baked, buggy implementation of a pre-emptive scheduler. We know that pre-emptive scheduling of isolated processes is a good abstraction. It's non-trivial to write a scheduler; certainly not something that every app developer should need to deal with.

I've said before and I'll say again: this kind of async is a design flaw. It's a retrograde step that brings all the cognitive overload of cooperative multitasking back. If the OS doesn't offer enough processes for some reason (e.g. too heavyweight) the answer should not be to throw our hands up and regress. The better answer is to work out how to provide more processes. It's absolutely doable: look at Erlang.

Of course things can improve still further; e.g. Dataflow variables [0] provide an effective abstraction for processing pipelines.

[0]: https://en.wikipedia.org/wiki/Oz_%28programming_language%29

I agree, and the docs got better too. In 3.7, asyncio is usable for people that don't understand it very well.

Before that, you had to learn the whole thing brick by brick before being able to do something serious.

Yet, there is a missing piece I'm hoping we'll see in 3.8: a way to limit the scope of `asyncio.ensure_future()`.

Indeed, right now either you `await` to get a sequential execution , or you call `asyncio.ensure_future()` to get a concurrent one. The later, unfortunatly, is the equivalent of a GOTO, and worse, it can contain a GOTO itself (see https://vorpus.org/blog/notes-on-structured-concurrency-or-g...).

So the best practice is to use `asyncio.gather()` to delimitate the pyramid of the task life cycle. Unfortunatly few people kow this, and hence do it. Plus it is not fun to do, it's boring boilerplate, something Python usually frees you of.

Yuri is thinking about how to implement the trio solution (the infamous nursery) in uvloop, and if he does, we usually get the feature ported to the stdlib a year later.

Meanwhile, I noted that a simple wrapper does meet the Pareto requirement: https://github.com/Tygs/ayo/

You can see it's not really hard to write your own version of it if you need to. It helped me a lot: the code is easier to reason about, and you remove a lot of edge cases.

I'll have to test Janus, it seems super nice.