Speed of Rust vs. C • Hacker News Board

Re: Speed of Rust vs. C

#41

Post by Jweb_Guru » Sat, Mar 13, 2021, 6:25 AM UTC

jandrewrogers wrote: post #29

oivey wrote: post #8

Earlier quoted context omitted.

> As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Not my experience at all. One big problem is that most languages in 2021 have very, very poor support for thread-based parallelism. It’s crazy how many languages make it hard to do basic data parallel tasks. That steers people toward wri…

Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limi…

There is not a meaningful semantic difference between what you're describing and what tools like rayon provide (and BTW, threads do just fine when pinned to a core and appropriately managed as they should be in large data processing workloads). Whether threads are used on the backend is largely a distraction, you still have to write things roughly the same way to create correct code (for example, you cannot share memory between tasks on different cores, or on different nodes, without synchronizing somehow).

Re: Speed of Rust vs. C

#42

Post by oivey » Sat, Mar 13, 2021, 6:27 AM UTC

IgorPartola wrote: post #20

oivey wrote: post #8

Earlier quoted context omitted.

> As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Not my experience at all. One big problem is that most languages in 2021 have very, very poor support for thread-based parallelism. It’s crazy how many languages make it hard to do basic data parallel tasks. That steers people toward wri…

In what way is process based parallelism strictly worse? On Linux a process and a thread in the kernel are both the same thing. The main difference between the two is that threads share memory by default whereas a process would need to explicitly mmap a chunk of memory to share with another process. This means that with threads you get to save RAM because of application code not having to take up more space (except m…

As pointed out in a sibling, processes have their own address spaces and aren’t as cheap to spawn as threads. I write code involving shared memory. It’s usable, but it also a pretty big pain to get right. It also significantly complicates things like managing memory ownership.

Re: Speed of Rust vs. C

#43

Post by otabdeveloper4 » Sat, Mar 13, 2021, 6:27 AM UTC

jandrewrogers wrote: post #29

oivey wrote: post #8

Earlier quoted context omitted.

> As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Not my experience at all. One big problem is that most languages in 2021 have very, very poor support for thread-based parallelism. It’s crazy how many languages make it hard to do basic data parallel tasks. That steers people toward wri…

Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limi…

> purpose-built scheduler + native coroutines

"Threads" are nothing but kernel-mode coroutines with purpose-built schedulers in the kernel.

Redoing the same machinery except in usermode is not the way to get performance.

The problem is that scripting languages don't let you access the kernel API's cleanly due to various braindead design decisions - global locks in the garbage collector, etc.

But the solution isn't to rewrite the kernel in every scripting language, the solution is to learn to make scripting languages that aren't braindead.

Re: Speed of Rust vs. C

#44

Post by jez » Sat, Mar 13, 2021, 6:31 AM UTC

trishume wrote: post #28

jandrewrogers wrote: post #3

As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Given this, it is not clear to me that Rust actually buys much when it comes to parallel programming for the purposes of performance. Is there another reason to focus on parallelism aside from performance? This reminds me of when I use to…

My impression is that what kind of parallelism patterns you need is pretty consistent within entire fields of programming. So you can go an entire career of performance optimization within HPC, game dev, film rendering or trading systems and never use the patterns the others say they use all the time. My experience with process-based parallelism is that yes on Linux it's basically isomorphic to thread-based paralleli…

> In Rust adding a new special-purpose background thread with some standard-library channels is 30 lines of code and I can probably even access the same logging system from the other thread.

Do you happen to have a link to code that does this? This sounds similar to a problem I have right now and I’d love to see what solution you’ve arrived at.

Re: Speed of Rust vs. C

#45

Post by Jweb_Guru » Sat, Mar 13, 2021, 6:32 AM UTC

andi999 wrote: post #35

jandrewrogers wrote: post #29

Earlier quoted context omitted.

Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limi…

Thanks. What is latency hiding?

Latency hiding is a a way to substantially increase throughput by queuing massive numbers of requests or tasks while waiting on expensive resources (e.g. main memory access can have latency in the hundreds of cycles for GPUs). By scheduling enough tasks or enough requests that can be dispatched in parallel (or very soon after one another), after an initial delay you may be able to process the queued requests very quickly (possibly once per clock cycle or two) providing similar overall performance to running the same set of tasks sequentially with very low latency. However, such long pipelines are very prone to stalling, especially if there are data dependencies that prevent loads from being issued early, so getting maximum performance out of code on architecture that heavily exploits latency hiding techniques can require a lot of very specific domain knowledge.

Re: Speed of Rust vs. C

#46

Post by pabs3 » Sat, Mar 13, 2021, 6:35 AM UTC

FTR, there are some efforts to integrate GCC & Rust:

https://github.com/antoyo/rustc_codegen_gcc https://github.com/Rust-GCC/gccrs https://github.com/sapir/gcc-rust/

Re: Speed of Rust vs. C

#47

Post by malkia » Sat, Mar 13, 2021, 6:45 AM UTC

trishume wrote: post #28

jandrewrogers wrote: post #3

As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Given this, it is not clear to me that Rust actually buys much when it comes to parallel programming for the purposes of performance. Is there another reason to focus on parallelism aside from performance? This reminds me of when I use to…

My impression is that what kind of parallelism patterns you need is pretty consistent within entire fields of programming. So you can go an entire career of performance optimization within HPC, game dev, film rendering or trading systems and never use the patterns the others say they use all the time. My experience with process-based parallelism is that yes on Linux it's basically isomorphic to thread-based paralleli…

The bigger issue is coordinating these threads ("workers") with threads from other processes, there is nothing on Windows and Linux to do so, then again I haven't had much experience with Grand Dispatch (OSX) to know if it's worth. Windows has new thread pool API, but even TBB or ConCRT do not use it. (though the new par-support in STL (msvc) does).

Re: Speed of Rust vs. C

#48

Post by CJefferson » Sat, Mar 13, 2021, 6:48 AM UTC

jandrewrogers wrote: post #29

oivey wrote: post #8

Earlier quoted context omitted.

> As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficiently process workloads. Not my experience at all. One big problem is that most languages in 2021 have very, very poor support for thread-based parallelism. It’s crazy how many languages make it hard to do basic data parallel tasks. That steers people toward wri…

Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limi…

I don't understand -- isn't what you are suggesting single threaded async code? That might be useful for servers, where you are mostly waiting for other things (like databases and networks), but in ithe places the point of parallel is to get all your CPUs doing useful work, and then (in my experience, happy to be shown counterexamples), coroutines aren't very useful. You just want to blast a bunch of threads (or rightly coupled processes)

Re: Speed of Rust vs. C

#49

Post by jandrewrogers » Sat, Mar 13, 2021, 6:48 AM UTC

andi999 wrote: post #35

jandrewrogers wrote: post #29

Earlier quoted context omitted.

Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limi…

Thanks. What is latency hiding?

There are many operations on data that are relatively slow from a CPU’s perspective — filling a cache line, page faulting, cache coherency, acquiring a lock, waiting on I/O to complete, etc. All of these add latency by stalling execution. In conventional software, when these events occur you simply stall execution, possibly triggering a context switch (which is very expensive). In many types of modern systems, these events are extremely frequent.

Latency hiding is a technique where 1) most workloads are trivially decomposed into independent components that can be executed separately and 2) you can infer or directly determine when any particular operation will stall. There are many ways to execute these high latency operations in an asynchronous and non-blocking way such that you can immediate work on some other part of the workload. The “latency-hiding” part is that the CPU is rarely stalled, always switching to a part of the workload that is immediately runnable if possible so that the CPU is never stalled and always doing real, constructive work. Latency-hiding optimizes for throughput, maximizing utilization of the CPU, but potentially increasing the latency of specific sub-operations by virtue of reordering the execution schedule to “hide” the latency of operations that would stall the processor. For many workloads, the latency of the sub-operations doesn’t matter, only the throughput of the total operation. The real advantage of latency-hiding architectures is that you can approach the theoretical IPC of the silicon in real software.

There are exotic CPU architectures explicitly designed for latency hiding, mostly used in supercomputing. Cray/Tera MTA architecture is probably the canonical example as well as the original Xeon Phi. As a class, latency-hiding CPU architectures are sometimes referred to as “barrel processors”. In the case of Cray MTA, the CPU can track 128 separate threads of execution in hardware and automatically switch to a thread that is immediately runnable at each clock cycle. Thread coordination is effectively “free”. In software, switching between logical threads of execution is much more by inference but often sees huge gains in throughput. The only caveat is that you can’t ignore tail latencies in the design — a theoretically optimal latency-hiding architecture may defer execution of an operation indefinitely.

Re: Speed of Rust vs. C

#50

Post by kelnos » Sat, Mar 13, 2021, 6:48 AM UTC

moonchild wrote: post #33

rectang wrote: post #18

Earlier quoted context omitted.

> ABI compatibility Rust provides ABI compatibility against its C ABI, and if you want you can dynamically link against that. What Rust eschews is the insane fragile ABI compatibility of C++, which is a huge pain to deal with as a user: https://community.kde.org/Policies/Binary_Compatibility_Issu... I don't think we'll ever see as comprehensive an ABI out of Rust as we get out of C++, because exposing that much incid…

Swift has a stable ABI. It makes different tradeoffs than rust, but I don't think complexity is the cliff. There is a good overview at https://gankra.github.io/blah/swift-abi/

Swift has a stable ABI at the cost of what amounts to runtime reflection, which is expensive. That doesn't really fit with the goals of Rust, I don't think.