Rust and C++ on Floating-Point Intensive Code
reidatcheson.com
Rust and C++ on Floating-Point Intensive Code
1–10 of 95 posts
Re: Rust and C++ on Floating-Point Intensive Code
#2So the difference basically boils down to -ffast-math, right? Is there an equivalent in Rust?
Edit: After some search I found these:
https://github.com/rust-lang/rust/issues/21690
https://doc.rust-lang.org/core/intrinsics/fn.fadd_fast.html
Writing a wrapper around f64 that uses these intrinsics shouldn't be too hard. I don't program in Rust though.
Re: Rust and C++ on Floating-Point Intensive Code
#3It's very easy to do FMA's using .mul_add() on floats in Rust, which the author didn't seem to know about.
Re: Rust and C++ on Floating-Point Intensive Code
#4I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply.
What I've found is that you need to ignore most of Rust: use/load raw pointers, don't use slices, unroll manually, vectorize manually, and check preconditions manually. You'll still get the amazing type system, but the code will have to be more C-like than Rust-like.
* raw pointers: LLVM is pretty good at optimizing C code. Rust specific optimization needs some work. (edit: I assumed arrays here, so you'll need the pointer for offsets; references are still okay. You'd also use the pointers for iterating instead of the usual slice iteration patterns)
* no slices: index checking is expensive, not to the CPU, the CPU rarely misses the check branches, but to the optimizer. I've found these are mostly left un-elided, even after inlining.
* no slices: slice indexing uses overflow checking. For Rav1e's case, the block/plane sizes mean that doing the index calculation using `u32` will never overflow, so calculating the offsets using u32 is fine (I'll have to switch to using a pseudo u24 integer for GPUs though, because u32 is still expensive on them).
* unroll manually: LLVM would probably do more of this with profiling info, but I've never found it (this is subjective!) to do any unrolling w/o. Maybe if all the other items here are also done...
* vectorize manually: Similar to unrolling. I've observed only limited automatic vectorization.
* And to get safety back: check, check, and check before calling the fast kernel! Ie wrap the kernel function with one that does all the checks elided in the kernel.
Source: Wrote https://github.com/xiph/rav1e/pull/1716, which speeds up the non-asm encodes by over 2x!
Re: Rust and C++ on Floating-Point Intensive Code
#5It's very easy to do FMA's using .mul_add() on floats in Rust, which the author didn't seem to know about.
Ideally the compiler should be able to do this by itself though, at least with the appropriate flag to enable it.
Re: Rust and C++ on Floating-Point Intensive Code
#6I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to igno…
Not sure I understand the part about raw pointers. As far as I understand, Rust references will surely turn into pointers at the LLVM IR level?
Re: Rust and C++ on Floating-Point Intensive Code
#7So the difference basically boils down to -ffast-math, right? Is there an equivalent in Rust? Edit: After some search I found these: https://github.com/rust-lang/rust/issues/21690 https://doc.rust-lang.org/core/intrinsics/fn.fadd_fast.html Writing a wrapper around f64 that uses these intrinsics shouldn't be too hard. I don't program in Rust though.
It doesn't exist yet and it's not clear it should be replicated as is. The fast-math flag does a bunch of related things that should probably be exposed separately so it's not a footgun in several situations. I'm also partial to exposing it per-function so the control is actually in the hands of the people writing the code that know the context and not subject to someone fiddling with compiler flags and getting incorrect code.
For this example you'd probably want -fassociative-math and not the other stuff that may result in incorrect code. -ffast-math was not actually used in the clang compilation and it's possible that the -fvectorize that was used picks a sensible mix of options.
Here's a preliminary discussion:
https://internals.rust-lang.org/t/pre-rfc-whats-the-best-way...
Trying to do an RFC process for this could be useful. The rust development process seems to be pretty good at thinking deeply about these things.
Re: Rust and C++ on Floating-Point Intensive Code
#8I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to igno…
Not sure I understand the part about raw pointers. As far as I understand, Rust references will surely turn into pointers at the LLVM IR level?
They do. The issue may be that references don't support pointer arithmetic – that is, given `x: &u32` you cannot get `x[1]`. The normal approach is to use a reference to a slice, `x: &[u32]`, and the default bounds-checked indexing. Rust does let you do unchecked indexing on slices (with `unsafe`), but it may be more efficient to avoid indices entirely in favor of pointer arithmetic. LLVM can often optimize index calculations into pointer arithmetic, but not always.
Edit: Although, on rereading the above post, I see diamondlovesyou did mention indices, so... not really sure what's going on.
Re: Rust and C++ on Floating-Point Intensive Code
#9It's very easy to do FMA's using .mul_add() on floats in Rust, which the author didn't seem to know about.
Ideally the compiler should be able to do this by itself though, at least with the appropriate flag to enable it.
If you’re doing fiddly numerical work, this must definitely be optional, as swapping separate multiplication and addition for FMA (or vice versa) can compromise correctness. In some cases you need two different algorithms if FMA is present or absent.
Re: Rust and C++ on Floating-Point Intensive Code
#10I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to igno…
Not sure I understand the part about raw pointers. As far as I understand, Rust references will surely turn into pointers at the LLVM IR level?
True. I should have explained that better. If you're accessing an array, you'll have to do the pointer offset-ing yourself (otherwise you're using a slice and all the checking that entails). Thus, you can't use a reference type, because reference types don't have `::add` like pointers do (also, `&T as usize` is invalid; you have to go through a pointer type first). I suppose I assumed you'd be accessing an array/slice; references otherwise are fine.