Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of Rust async and Linux thread context switch time and memory use

github.com

Post reply on HN
Print view View original

141–150 of 201 posts

Re: Comparison of Rust async and Linux thread context switch time and memory use

#141

Post by Serow225 »

moocowtruck wrote: post #132
lxpz wrote: post #119

Earlier quoted context omitted.

That .NET syntax using Task.WhenAll seems quite bad, which might be part of the reason why not many people bother (disclaimer: I don't do C# or ASP.NET). In Rust it would be: let p, c, m = join!( GetUserPermission(username), GetServerConfig(), GetMessageOftheDay() ); (you don't even have to write await when using the join macro) With such simple syntax available it seems obvious to me that one would want to use it as…

var r = await Task.Whenall(f1,f2,f3); Console.WriteLine($"{r[0]}, {r[1]}, {r[2]}"); f1,f2,f3 are all async fn's, that is all you have to do

yeah, the original example is showing the unwieldy version of the syntax.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#142

Post by mikeschmatz »

GordonS wrote: post #106
jiggawatts wrote: post #88

Earlier quoted context omitted.

Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages. That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode): var p = await GetUserPermission( username ); var c = await GetServerConfig(); var m = await GetMessageOfTheDay(); Assume each await call is…

I've been using C# for around 20 years, basically since it was first released. I never personally had any issue with working with threads and locks, finding it simple enough to reason about them, though I understand lots of people felt differently. When async/await first came to C# around 10 years ago, I grumbled because I didn't see the point; I found it much harder to reason about the flow of code, and initially at…

> Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint.

AFAIK, .Net doesn't support "green threads" and they repeatedly confirmed that there are no future plans to do so. Additionally, M:N threading model has serious interop issues as evident in Go, which is a no-go for system languages. Personally, I don't see a need for green threads since kernel threads are fast enough and don't use that much RAM as people tend to believe. And when they are not, sure, go async/await.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#143

Post by scottlamb »

gwbas1c wrote: post #136

TLDR: Async code will have much lower CPU utilization compared to threaded code . An async version of a program might run just as fast as a threaded one, but it will overall use less system resources . The threaded version will be easier to write. You can also have lower RAM overhead per thread if you choose a smaller stack space. Many programs will run fine with a smaller stack space, BTW. ---- Years ago I had to bu…

> You can also have lower RAM overhead per thread if you choose a smaller stack space.

No, this is about as low as it gets. As the author explained, "the kernel only allocates physical memory to a stack as the thread touches its pages, so the initial memory consumption of a thread in user space is actually only around 8kiB."

The smallest possible page size (on x86-64) is 4 KiB, and you can't share pages between thread stacks, [1] so you can't go below 4 KiB of physical memory usage per thread. I'm not exactly sure how the author got to 8 KiB; maybe they meant "for each userspace thread" rather than "memory used in userspace" and are counting kernel memory too. I'm pretty sure the kernel uses at least 4 KiB per userspace thread (for a stack of its own, among other overhead).

Green threads won't take you below 4 KiB either, for the same reason.

[1] Without some custom ABI that guards against stack overflow in a different way. Golang has a custom ABI (I'm not sure exactly if this is why), and interoperability with C suffers, so this isn't an approach I'd love for Rust.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#144

Post by Freaky »

Ciantic wrote: post #83
josephg wrote: post #53

Earlier quoted context omitted.

I’d happily take an async-by-default world over a world where some APIs only exist through blocking calls. A classically threaded program can easily block on a future, but wrapping a blocking call in an otherwise asynchronous program is complicated, expensive and error prone work.

Btw, have you read this: https://async.rs/blog/stop-worrying-about-blocking-the-new-a... async-std allows to run blocking calls without hoops rather efficiently: async fn read_to_string(path: impl AsRef ) -> io::Result { std::fs::read_to_string(path) } It doesn't have await inside! My mind was blown as I saw that.

Read the first sentence.

> This blog post describes a proposed scheduler for async-std that did not end up being merged for several reasons.

I don't think it's a particularly good idea in the first place - it's basically an automatic watchdog-driven block_in_place(). It doesn't remove the problem of blocking in futures, it just limits the damage to the local task rather than blocking the entire executor.

That's fine in the simple case of future-per-task, but it's pretty common to be polling multiple futures concurrently within one, so it's not a general solution.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#145

Post by the8472 »

bitcharmer wrote: post #112
the8472 wrote: post #95

Earlier quoted context omitted.

I have done benchmarking on a linux laptop. Once you disable turbo-boost you get quite consistent results for CPU-bound tasks at least.

You have to do so much more to be able to reliably measure events on the scale of nanos. You need to lock C-states, disable P-state driver, isolate CPUs, get rid of RCUs, affinitize your tasks, enable low-tick mode, skew hr ticks, make sure you use TSC clocksource, set the cpu governor, get rid of vmstat, set correct idle driver, disable audits, and watchdogs and much, much more.

If you want to instrument only a handful events, yes. But for microbenchmarks which you can run for many iterations to get min/max/stdev (such as the benchmarks in the article) it's much easier. Disabling turbo often is sufficient to lower the variance far enough that old and new code are clearly distinguishable.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#146

Post by GordonS »

mikeschmatz wrote: post #142
GordonS wrote: post #106

Earlier quoted context omitted.

I've been using C# for around 20 years, basically since it was first released. I never personally had any issue with working with threads and locks, finding it simple enough to reason about them, though I understand lots of people felt differently. When async/await first came to C# around 10 years ago, I grumbled because I didn't see the point; I found it much harder to reason about the flow of code, and initially at…

> Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint. AFAIK, .Net doesn't support "green threads" and…

TIL: I've been using the term "green threads" incorrectly for years! [0]

I had actually meant "normal", OS-level threads.

[0] https://stackoverflow.com/a/42454139/25758

Re: Comparison of Rust async and Linux thread context switch time and memory use

#147

Post by jeremyjh »

justsomeuser wrote: post #138
IshKebab wrote: post #65

Earlier quoted context omitted.

That's the most commonly given reason for using async/await so a lot of people assume threads are way more heavyweight than they actually are. What other reason were you thinking of?

I find async/await to be easier to use than alternatives. I see two categories: - A. Async/await - compiler saves and resumes functions. - B. Message based - Golang, Erlang, threads with messaging. With category A, I can use my IDE to jump to every function that is called and easily follow the computation. With category B, all of these connections happen at runtime with messages. When you have a tree of tasks all whi…

This a confused notion. A useful way to think of Go and Erlang is that they automatically and transparently insert async/await each time you call a function that performs I/O. Messaging between different application tasks is completely orthogonal and can have use cases in languages with async/await as well.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#148

Post by _vvhw »

volta83 wrote: post #114
_vvhw wrote: post #79

Earlier quoted context omitted.

Where 1 iteration and 1 syscall to io_uring_enter() is submitting 100s of I/O operations per io_uring_enter() syscall (and you can even run the ring buffers with the kernel set to poll so you can do 0 syscalls if that's not already enough). That's pretty huge amortization. Rough benchmarks we've done are showing double throughput for io_uring for 4096 byte AF sector write/fsync/read combos: https://github.com/coilhq/…

> Where 1 iteration and 1 syscall to io_uring_enter() is submitting 100s of I/O operations per io_uring_enter() syscall (and you can even run the ring buffers with the kernel set to poll so you can do 0 syscalls if that's not already enough). Same for the blocking case. If I do a syscall to read a whole file, its just 1 syscall creating millions of I/O operations.

Sure, but that's all you'll ever do with the blocking case: 1 syscall at a time, while your program sits and does nothing with the CPU, whereas with io_uring at least you can do CPU while you wait on your IO. So even ignoring the IORING_SQPOLL option that requires no io_uring_enter() syscall, a basic usage of io_uring is still going to be faster.

io_uring is a bicycle for IO, and you can ride it as fast as you want to. But it's apples and oranges to blocking IO, which is always stuck in first gear.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#149

Post by mikeschmatz »

kohlerm wrote: post #59

To me it looks like the main advantage of async is memory usage, which is kind of expected because of the overhead of a thread. But if you do not need lots of thread it doesn't look like there is a huge benefit going async. Or do I miss something here?

It was my conclusion as well. I only found async useful in situations when a service has to deal with a large number of incoming requests, e.g. web server

Re: Comparison of Rust async and Linux thread context switch time and memory use

#150

Post by Freaky »

masklinn wrote: post #86
Ciantic wrote: post #83

Earlier quoted context omitted.

Btw, have you read this: https://async.rs/blog/stop-worrying-about-blocking-the-new-a... async-std allows to run blocking calls without hoops rather efficiently: async fn read_to_string(path: impl AsRef ) -> io::Result { std::fs::read_to_string(path) } It doesn't have await inside! My mind was blown as I saw that.

> should a task execute for too long, the runtime will automatically react by spawning a new executor thread taking over the current thread’s work. That is a super interesting strategy, though obviously only works when you can « afford » a multithreaded scheduler. Anyway I wonder how they manage this, signals?

Nothing so fancy.

Each worker thread runs in a loop executing a queue of jobs. On every iteration it sets an atomic progress flag to true.

The runtime in which it's contained polls its workers every 1-10ms, atomically swapping in false and checking to see if the previous value was also false - if so, it steals its task queue and spins up another worker to execute it.

https://github.com/async-rs/async-std/blob/ceba324bef9641d61...

Post reply on HN