Comparison of Rust async and Linux thread context switch time and memory use • Hacker News Board

Re: Comparison of Rust async and Linux thread context switch time and memory use

#161

Post by jayd16 » Fri, Feb 12, 2021, 5:11 PM UTC

jiggawatts wrote: post #88

lxpz wrote: post #75

> People often see that there's some theoretical benefit of async and then they accept far less ergonomic coding styles and the additional bug classes that only happen on async due to accidental blocking etc... despite the fact that when you consider a real-world deployed application, those "benefits" become indistinguishable from noise. However, due to the additional bug classes and worse ergonomics, there is now le…

Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages. That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode): var p = await GetUserPermission( username ); var c = await GetServerConfig(); var m = await GetMessageOfTheDay(); Assume each await call is…

You don't need to create a Task[] because WhenAll is set up for varargs. This is fine:

    await Task.WhenAll(p_t, c_t, m_t);

Or you can just await the threads before you need them. They're already started and running at this point.

You also probably want to avoid using Result and just await the completed task for the nicer unwrap syntax. Plus, you don't want to get into the habit of using Result as its a blocking call. Same with WaitAll and WaitAny. Ideally you would never use those. ContinueWith is also not very needed if your style is to use the more plain await syntax. Those methods are more to bridge blocking and async code so an async from the start app might use async extensively and never those methods.

Perhaps search for WhenAny and WhenAll?

Re: Comparison of Rust async and Linux thread context switch time and memory use

#162

Post by bzbarsky » Fri, Feb 12, 2021, 5:18 PM UTC

johnsoft wrote: post #155

bzbarsky wrote: post #122

Earlier quoted context omitted.

Are your users going to be running your application on laptops? Will they have the same "conserve power by limiting performance" going on? If so, that is _exactly_ the environment you want to do performance work in, generally speaking.

It's about having a consistent measurement baseline. Say you run your benchmark once, then thermal throttling kicks in, then you run it again, and it takes twice as long. Is your code actually slower now? Should I wait until the fan turns off before I run it again? That data is noisy and useless. Take your measurements on a server or desktop with sane thermals and a full-size fan. If you speed things up by 10% on you…

Yes, you have to be very careful with measurements, I agree.

> If you speed things up by 10% on your server, they'll get 10% faster on your laptop as well.

Depends on the speedup and techniques to achieve it. For example, speeding things up via more parallelism can lead to wall-clock improvements on servers but not laptops, precisely because the latter just end up doing more thermal throttling....

Ideally, you want to measure both ideal hardware and actual-user-hardware; often speedups on one will not be visible on the other and vice versa.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#163

Post by _vvhw » Fri, Feb 12, 2021, 5:22 PM UTC

volta83 wrote: post #152

_vvhw wrote: post #148

Earlier quoted context omitted.

Sure, but that's all you'll ever do with the blocking case: 1 syscall at a time, while your program sits and does nothing with the CPU, whereas with io_uring at least you can do CPU while you wait on your IO. So even ignoring the IORING_SQPOLL option that requires no io_uring_enter() syscall, a basic usage of io_uring is still going to be faster. io_uring is a bicycle for IO, and you can ride it as fast as you want t…

> while your program sits and does nothing with the CPU The CPU can run other threads while the hardware does DMA transfers. The thread just yields when the transfer is started, and a hardware exception wakes it up when the DMA transfer finishes.

Sure, but we're comparing the efficiency of one of your program's single threads, because otherwise you could take that same argument you just used and turn it around and say fine, just run another thread then with another io_uring... and you're still ahead. You have to compare at the smallest unit of control plane.

At the same time, multiple threads for a single program introduce context switches which are becoming horrendously expensive compared to the sheer number of IOPS that modern NVMe SSDs can do.

Thread-per-core designs built around io_uring are the future of IO on Linux.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#164

Post by bzbarsky » Fri, Feb 12, 2021, 5:22 PM UTC

jstrong wrote: post #125

bzbarsky wrote: post #122

Earlier quoted context omitted.

Are your users going to be running your application on laptops? Will they have the same "conserve power by limiting performance" going on? If so, that is _exactly_ the environment you want to do performance work in, generally speaking.

generally speaking, the advantage of async io is strongest for high performance server applications, especially in regards to the cpu usage required relative to the amount of io stuff you can do. with that in mind, "users running your application on laptops" would not be the most common case.

Yes, if your app is a high performance server app, measure in that environment.

But user-facing apps (the sort people run on laptops, say) have async I/O as table stakes, really. It's not even about throughput or CPU cycles: it's about the fact that if you have I/O latency on any thread the user interacts with the user experience will be terrible.

Now in practice maybe that means "just make the I/O async, but the performance details of that don't really matter too much".

Anyway, the overall comment was about performance profiling in general, not just async I/O.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#165

Post by pow_pp_-1_v » Fri, Feb 12, 2021, 5:27 PM UTC

GordonS wrote: post #106

jiggawatts wrote: post #88

Earlier quoted context omitted.

Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages. That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode): var p = await GetUserPermission( username ); var c = await GetServerConfig(); var m = await GetMessageOfTheDay(); Assume each await call is…

I've been using C# for around 20 years, basically since it was first released. I never personally had any issue with working with threads and locks, finding it simple enough to reason about them, though I understand lots of people felt differently. When async/await first came to C# around 10 years ago, I grumbled because I didn't see the point; I found it much harder to reason about the flow of code, and initially at…

async/await is for the concurrent stuff and threads are for the parallel stuff. Two different things. If your code is I/O-bound, use async/await. If your code is processor-bound thing, use threads.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#166

Post by xfer » Fri, Feb 12, 2021, 5:29 PM UTC

lxpz wrote: post #126

ceronman wrote: post #90

Earlier quoted context omitted.

It seems to me that these are two orthogonal topics. One thing is how you represent tasks, either using OS threads or async tasks. And the other is how you structure concurrency. Maybe I'm missing something, but I think there is nothing preventing the use of those structured concurrency patterns using OS threads as the base for tasks. Then you get some nice benefits of doing this such as proper stack-traces and easie…

The advantage of async tasks for structured concurrency lies in task cancellation, which is intrinsically linked to the notion of "task ownership". If you are using an OS thread to offload some task, and then realize that you don't need that task's result anymore, your safest bet is to let the thread run until the end and then discard the results it produces. Other options include adding custom cancellation logic to…

All haskell threads are cancellable. This does mean you have to take extra care when using certain constructs.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#167

Post by reddit_clone » Fri, Feb 12, 2021, 6:09 PM UTC

waynesonfire wrote: post #99

lxpz wrote: post #75

> People often see that there's some theoretical benefit of async and then they accept far less ergonomic coding styles and the additional bug classes that only happen on async due to accidental blocking etc... despite the fact that when you consider a real-world deployed application, those "benefits" become indistinguishable from noise. However, due to the additional bug classes and worse ergonomics, there is now le…

It has been a pleasure using the Erlang runtime to scratch my concurrency itch while avoiding the async / await bandaid.

Seemingly synchronous on the inside. Async on the outside. With (nearly/practically) unlimited processes/greenthreads.

Best combination of things I have come across.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#168

Post by mst » Fri, Feb 12, 2021, 6:10 PM UTC

makapuf wrote: post #58

jstrong wrote: post #40

Earlier quoted context omitted.

ah, a fellow traveler - godspeed. what a sane, reasonable world we could have if nanosecond timestamps ruled supreme.

I wouldn't be so extreme, providers of '0' keys would flourish

I occasionally wonder whether it would be easier to skim '1k us' than '1ms' though, providing everything was denominated in us.

Maybe I'll try it in a blog post one day and see what percentage of the comments consist of hurled fruit.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#169

Post by lostcolony » Fri, Feb 12, 2021, 6:19 PM UTC

ceronman wrote: post #90

lxpz wrote: post #75

> People often see that there's some theoretical benefit of async and then they accept far less ergonomic coding styles and the additional bug classes that only happen on async due to accidental blocking etc... despite the fact that when you consider a real-world deployed application, those "benefits" become indistinguishable from noise. However, due to the additional bug classes and worse ergonomics, there is now le…

It seems to me that these are two orthogonal topics. One thing is how you represent tasks, either using OS threads or async tasks. And the other is how you structure concurrency. Maybe I'm missing something, but I think there is nothing preventing the use of those structured concurrency patterns using OS threads as the base for tasks. Then you get some nice benefits of doing this such as proper stack-traces and easie…

So one of the things I realized writing Erlang is that when concurrency is 'free' (or so close as to be indistinguishable in most use cases), more things end up being easy to write concurrently than we traditionally think.

An instance I ran into personally was, effectively, task scheduling. Sure, I could have done the 'normal' thing, of a priority queue being populated from the database on some interval, having some thread reading from that queue, sleeping until the first item needs work, pulling it off, throwing it onto a threadpool. Have to take care to ensure the threadpool is large enough for the maximum amount of concurrency I need, have to make sure that I'm careful in what data structure I use for the priority queue (I need to make sure I'm not adding the same task multiple times to it, and that when adding items to it I'm not locking it), make sure the polling thread can't throw (or at least, when it does, it restarts or kills the program and that then restarts), a few other niggles here and there too. And a whole 'nother level of complexity if tasks lead to follow up tasks (i.e., a task represents a state machine through a series of transitions, which themselves take a sizable amount of time, to where just leaving them on the thread is a bad idea, since it uses up the threadpool).

In a 'free concurrency' world, I just spin up a new concurrent process per task for some window (same as how many items I added to the priority queue). And that's basically it. Each process can step through its state machine, sleeping in between tasks for however long, without issue.

Re: Comparison of Rust async and Linux thread context switch time and memory use

#170

Post by bitcharmer » Fri, Feb 12, 2021, 6:27 PM UTC

the8472 wrote: post #145

bitcharmer wrote: post #112

Earlier quoted context omitted.

You have to do so much more to be able to reliably measure events on the scale of nanos. You need to lock C-states, disable P-state driver, isolate CPUs, get rid of RCUs, affinitize your tasks, enable low-tick mode, skew hr ticks, make sure you use TSC clocksource, set the cpu governor, get rid of vmstat, set correct idle driver, disable audits, and watchdogs and much, much more.

If you want to instrument only a handful events, yes. But for microbenchmarks which you can run for many iterations to get min/max/stdev (such as the benchmarks in the article) it's much easier. Disabling turbo often is sufficient to lower the variance far enough that old and new code are clearly distinguishable.

It has nothing to do with instrumenting and everything to do with platform noise.