Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of Rust async and Linux thread context switch time and memory use

github.com

Post reply on HN
Print view View original

151–160 of 201 posts

Re: Comparison of Rust async and Linux thread context switch time and memory use

#151

Post by spankalee »

lxpz wrote: post #126
ceronman wrote: post #90

Earlier quoted context omitted.

It seems to me that these are two orthogonal topics. One thing is how you represent tasks, either using OS threads or async tasks. And the other is how you structure concurrency. Maybe I'm missing something, but I think there is nothing preventing the use of those structured concurrency patterns using OS threads as the base for tasks. Then you get some nice benefits of doing this such as proper stack-traces and easie…

The advantage of async tasks for structured concurrency lies in task cancellation, which is intrinsically linked to the notion of "task ownership". If you are using an OS thread to offload some task, and then realize that you don't need that task's result anymore, your safest bet is to let the thread run until the end and then discard the results it produces. Other options include adding custom cancellation logic to…

Anywhere you have an .await in async code you could have a checkpoint in a thread that allows for cancellation. That's the main cancellation advantage - that the author is forced to write those to consume other async functions.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#152

Post by volta83 »

_vvhw wrote: post #148
volta83 wrote: post #114

Earlier quoted context omitted.

> Where 1 iteration and 1 syscall to io_uring_enter() is submitting 100s of I/O operations per io_uring_enter() syscall (and you can even run the ring buffers with the kernel set to poll so you can do 0 syscalls if that's not already enough). Same for the blocking case. If I do a syscall to read a whole file, its just 1 syscall creating millions of I/O operations.

Sure, but that's all you'll ever do with the blocking case: 1 syscall at a time, while your program sits and does nothing with the CPU, whereas with io_uring at least you can do CPU while you wait on your IO. So even ignoring the IORING_SQPOLL option that requires no io_uring_enter() syscall, a basic usage of io_uring is still going to be faster. io_uring is a bicycle for IO, and you can ride it as fast as you want t…

> while your program sits and does nothing with the CPU

The CPU can run other threads while the hardware does DMA transfers. The thread just yields when the transfer is started, and a hardware exception wakes it up when the DMA transfer finishes.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#153

Post by harikb »

scoutt wrote: post #74
harikb wrote: post #32

Earlier quoted context omitted.

Keep in mind that a new async task doesn't create a new thread. So yes, "not creating a new thread" is 3x faster than "creating a thread". If the app layer can context switch using language level constructs, and do co-operative switching, then yes, one gets the 3x benefit. imho, whether the async executor and scheduler is performant enough to manage the tasks is what one should worry about.

I'm confused. If many async tasks are ran on a single thread, what the thread does when is blocked waiting for things to happen? Does it sleep? If so, a context switch takes place anyway. If not, what is the impact on GUI applications? If I have a main thread to manage my GUI, should I spin a new thread to run my async tasks? A modern microcontroller/microprocessor is inherently event driven (for example, on ARM, at…

polling is only explained as a logical thing. In reality the given task is only marked to be woken up later. The later being some other point, while the same OS thread executing something else, when the executor determines that the idling task can be woken up. "Waking" is nothing but the same OS thread now switching to execute whatever it is that it is waking up.

Main idea is that a 'scheduler/executor' at the runtime/language level that knows about the state of the program can (a) 'save' and 'restore' fewer things compared to an OS context switch. (b) co-operative stuff does not need to pay the cost of too many unnecessary pre-emptions

Re: Comparison of Rust async and Linux thread context switch time and memory use

#155

Post by johnsoft »

bzbarsky wrote: post #122
secondcoming wrote: post #69

Performance profiling on a laptop is largely pointless. There's too much stuff trying to conserve power by limiting performance.

Are your users going to be running your application on laptops? Will they have the same "conserve power by limiting performance" going on? If so, that is _exactly_ the environment you want to do performance work in, generally speaking.

It's about having a consistent measurement baseline. Say you run your benchmark once, then thermal throttling kicks in, then you run it again, and it takes twice as long. Is your code actually slower now? Should I wait until the fan turns off before I run it again? That data is noisy and useless. Take your measurements on a server or desktop with sane thermals and a full-size fan.

If you speed things up by 10% on your server, they'll get 10% faster on your laptop as well.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#156

Post by imtringued »

jstrong wrote: post #123
richardwhiuk wrote: post #84

The problem with threads is you need to correctly size your thread pool. That's hugely difficult if you have unknown lengths of blocking IO.

what kind of problem were you trying to solve that you found sizing a thread pool to be difficult? generally when I've worked on high performance server code I've been coding with a target machine in mind, so it's more a matter of mapping the thread pool size to the resources available on that machine. but I'm interested to hear about circumstances where it wouldn't be easy.

You have a threadpool with X threads. You dispatch Y tasks. X of them run for 5 minutes. That means the remaining Y-X tasks are delayed by 5 minutes despite low CPU utilization.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#157

Post by Serow225 »

jiggawatts wrote: post #88
lxpz wrote: post #75

> People often see that there's some theoretical benefit of async and then they accept far less ergonomic coding styles and the additional bug classes that only happen on async due to accidental blocking etc... despite the fact that when you consider a real-world deployed application, those "benefits" become indistinguishable from noise. However, due to the additional bug classes and worse ergonomics, there is now le…

Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages. That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode): var p = await GetUserPermission( username ); var c = await GetServerConfig(); var m = await GetMessageOfTheDay(); Assume each await call is…

My teams/company uses it all over, so maybe depends on the context you work in?

And FWIW, this explicit form is often unnecessary - if you kick off each task they will run in parallel and then just await each task only when the result is needed, it can look a lot cleaner:

        var p_t = GetUserPermission(username);
        var c_t = GetServerConfig();
        var m_t = GetMessageOfTheDay();

        var foo = isAuthorized(await p_t);
        // more code here
        var msg = ( (await c_t).ServerName + await m_t) );

Re: Comparison of Rust async and Linux thread context switch time and memory use

#158

Post by jerf »

lxpz wrote: post #75

> People often see that there's some theoretical benefit of async and then they accept far less ergonomic coding styles and the additional bug classes that only happen on async due to accidental blocking etc... despite the fact that when you consider a real-world deployed application, those "benefits" become indistinguishable from noise. However, due to the additional bug classes and worse ergonomics, there is now le…

I think the more important aspect of that quote is just about performance vs. code. I see many cases where people are hyperoptimizing on whether or not their framework consumes 2 or 15 microseconds per request when the work they are going to do takes 100 milliseconds.

If you like the async style better, then fine, use it. Sometimes you win like that, where the thing you like better is also faster. But don't worry so much about the performance.

Web frameworks is another place I see this a lot. Crossing the streams, if you've got an incoming web request, unless your framework somehow consumes and discards the web headers, a real web request is already many kilobytes just to represent the incoming headers by the time it gets to your handling code. Using async because it has ~200 bytes per task vs a thread allocating 10K out of the box at that point doesn't make much difference because the HTTP request itself is blowing out the difference.

The spread in orders of magnitude in what is expensive and what is not has gotten so significant on modern systems that you can easily get developers sitting there optimizing nanoseconds while throwing away seconds. The old school assembly-style premature optimization where we're trying to save every bit and cycle has mostly passed away, but its replacement seems to be this; frantically benchmarking how many millions of requests per second some framework or feature can handle as if it matters when your code is going to take 500ms.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#159

Post by scoutt »

harikb wrote: post #153
scoutt wrote: post #74

Earlier quoted context omitted.

I'm confused. If many async tasks are ran on a single thread, what the thread does when is blocked waiting for things to happen? Does it sleep? If so, a context switch takes place anyway. If not, what is the impact on GUI applications? If I have a main thread to manage my GUI, should I spin a new thread to run my async tasks? A modern microcontroller/microprocessor is inherently event driven (for example, on ARM, at…

polling is only explained as a logical thing. In reality the given task is only marked to be woken up later. The later being some other point, while the same OS thread executing something else, when the executor determines that the idling task can be woken up. "Waking" is nothing but the same OS thread now switching to execute whatever it is that it is waking up. Main idea is that a 'scheduler/executor' at the runtim…

Thanks.

> polling is only explained as a logical thing

But there is the poll() function that returns either the result of the operation, or "pending". So it's more than logical. Correct? I mean, if I (or the executor) don't call poll() nothing happens...

> OS thread now switching to execute whatever it is that it is waking up.

This is what confuses me. As I see it (and I what I understand from reading), async/await splits a routine into a (very smart) state machine.

I assume that there is no magic underneath. I mean, I can do the same state machine by hand if I want to, under the constraints of what the OS makes available for me in what context switching regards (APIs for waiting and synchronizing).

For a (OS/native) thread that has to wait for data on a socket, you have (basically) two options: wait on recv() or poll recv() without timeout.

Waiting on recv() would block (so no other code of my thread can be executed while waiting), so I guess the state machine needs to poll on recv() (I believe this is what this[1] example does).

In order to no block my thread, the executor either spins its own thread, or has to wait for my thread to poll() it.

[1] https://rust-lang.github.io/async-book/02_execution/02_futur...

Re: Comparison of Rust async and Linux thread context switch time and memory use

#160

Post by PixelOfDeath »

pron wrote: post #89

The throughput increase in I/O scenarios with many tasks is due to the number of supported concurrent processes and Little's law; it has little to do with context switching time, which has a negligible impact on the throughput in these use-cases: https://inside.java/2020/08/07/loom-performance/ Low context switch latency only matters when the number of tasks is very small (their data all fits in the cache), and the w…

I hat very good experience with using buffer variables to copy-"prefetch" unpredictable costly fetches. E.g. from cache lines that get touched by several cores for communication.

And I only actual use them after one iteration of whatever I do. So the core could fetch the memory content without actually having to stall because I do not use it until later.

I'm not sure how realistic that is inside a kernel thread scheduler, but it sure is useful in user space for task based libraries.

Post reply on HN