Comparison of Rust async and Linux thread context switch time and memory use

I see, I did not know that.

Last time I used Erlang (pre-Elixir), the `bytes` example would require you to set up a request/response with a blocking `receive`.

I feel like this analysis is missing some more nuanced points about stack memory.

Yes, pages will only be allocated for a thread's stack when the thread actually uses them. However, the thread does not release said memory afterwards. The memory can only be reused by the same thread. If a thread ever once does something that temporarily allocates a bunch of stack space, then it forever consumes that space going forwards even when no longer needing it. If you have 10,000 threads and each one of them happens to, at some point in its lifetime, use 1MB of stack space and frees it, then you are now using 10GB of RAM on mostly-unused pages.

Now you might say "what on Earth would ever use 1MB of stack???", but the problem is, in normal programs with few threads, there's no problem with a function temporarily using a ton of stack, and so random things feel free to do so. Maybe some library call you make likes to allocate a temporary buffer on the stack and you don't even know it. There's also normally no problem with doing some deep recursion every now and then, so it happens. Often, stack allocation is data-dependent (e.g. recursive descend parsing). So if you try to strictly limit your stack space then you risk running into random stack overflows or maybe even security issues. And if you do find a limit that works, it's still probably much larger than the average usage, so you're still wasting a bunch of memory.

IIRC, Go uses segmented stacks to avoid this problem, but C/C++/Rust do not. (I think Rust tried to at one point, but later gave up on that because of the complexity?)

In contrast, async tasks only hold onto the memory they are actually using to store live data at any particular moment. If an async task invokes some deeply-nested function and uses a bunch of stack space, it doesn't really matter, because all the tasks are running on the same thread, so the next task to call that function uses the same pages rather than allocate new ones.

(There's actually a similar issue regarding heap space. Memory allocators that perform reasonably with multiple threads typically maintain per-thread freelists, so if you have lots and lots of threads, you end up with a bunch of free'd memory stuck in freelists. Though, some allocators, like the new tcmalloc, are starting to use per-core freelists instead, which may avoid this problem.)

> Async programming also allows for structured programming, where each task is an owned resource of a parent tasks, which means that lifetimes of tasks can be controlled and runaway threads can't exist

You're confusing mechanism with semantics. Here's an article on how Java's Project Loom achieves structured concurrency using its new virtual threads capability: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

And the article cited above that proposes the idea of nurseries for structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

Arguably, structured concurrency as described above is easier to obtain when using threads as your underlying mechanism, because the vast majority of code is serial[1]. That there are a handful of critical regions where you want to express concurrency relationships doesn't mean we have to discard threads. That's throwing the baby out with the bath water.

Self-promotion: I had stumbled on the idea of "nurseries", independently and many years before the above were published. See https://github.com/wahern/cqueues It's nominally a non-blocking "threading" API for Lua. (In Lua coroutines are also called threads.) But note the plural, continuation queues. It's trivial to instantiate a queue, which is similar to a nursery. This was by design. Many cqueues projects naturally end up with a tree of thread controllers/schedulers. It doesn't work on Windows (yet) because it relies on the fact that kqueue, epoll, and Solaris Ports descriptors can be recursively polled.

[1] Serial != synchronous/blocking.

It certainly is easy to make a mess of it with C/C++. It can be done well and safely but there are no guard rails. I have written this code in C and trusted it to run as intended and it did. I wouldn’t stake human lives on it but that wasn’t my requirement at the time. Val grind and other code analysis tools certainly didn’t complain and I had no memory leaks. Rust didn’t exist at the time. One specific project wrangled about 1000 worker threads, a logging thread, a network server thread, a signal processing thread, and a main control thread to the tune of a very large number of requests per second on commodity hardware. In running it for I think 4 years I had one memory leak initially that Valgrind quickly found. Could probably write that service with a lot fewer LOCs today with a language like Rust of course and with all kinds of memory safety. But at the time it worked well. Oh and it had to do all kinds of fun low level networking stuff with elevated privileges so double danger :)

These days it is possible to eliminate almost all blocking calls in Linux apps. File opening was a long persisting one but io_uring fixes that. Async sockets and file i/o to already-open fd's have been around forever.