Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust and C++ on Floating-Point Intensive Code

reidatcheson.com

Post reply on HN
Print view View original

81–90 of 95 posts

Re: Rust and C++ on Floating-Point Intensive Code

#81

Post by nwallin »

paulddraper wrote: post #32
magicalhippo wrote: post #5

Earlier quoted context omitted.

Ideally the compiler should be able to do this by itself though, at least with the appropriate flag to enable it.

FMA isn't a safe optimization as it can give different results. C++ compilers have flags to enable it globally. gcc and clang include the optimization in -Ofast. Rust allows you to choose at a code level (but usually people don't know about it). Perhaps it should also have a global fast-math flag that would automatically optimize it. Pros and cons to that.

FMA is "safe" in that if it breaks your code, it was already broken. It can only make the results slightly more accurate, unlike for instance the rsqrt instruction which is less accurate. (and as such is not a safe optimization)

GCC emits FMA instructions at -O2 without -ffast-math.

Re: Rust and C++ on Floating-Point Intensive Code

#82

Post by steveklabnik »

vkaku wrote: post #78

Okay, can someone give an explanation for why Rust does not mimic the -O fast behavior? Is this something they plan to add?

It leads to undefined behavior in safe code in the general case.

We may add a wrapping type, similar to what we do for integer behavior. But in general, adding flags to change major behavior is not something we do.

Re: Rust and C++ on Floating-Point Intensive Code

#83

Post by mratsim »

magicalhippo wrote: post #37
paulddraper wrote: post #32

Earlier quoted context omitted.

FMA isn't a safe optimization as it can give different results. C++ compilers have flags to enable it globally. gcc and clang include the optimization in -Ofast. Rust allows you to choose at a code level (but usually people don't know about it). Perhaps it should also have a global fast-math flag that would automatically optimize it. Pros and cons to that.

I wasn't trying to imply it should be on by default. Often one does not care about the lower bits of the floats, but do want the speed. For some tasks it's very much the opposite. Being able to specify a global option with local override is a great combo.

FMA is not always faster, it has a high latency: 5-6 cycles depending on the CPU while Add and MUL have very low-latency.

This means that to fully utilizes FMA you need to unroll a loop more. Sometimes yyou just can't, and the other time you use more instructions, use more cache.

In short it's not always better.

Also as other said, FMA has better accuracy than separate Add + Mul

Re: Rust and C++ on Floating-Point Intensive Code

#84

Post by silentvoice »

tom_mellior wrote: post #67
lovasoa wrote: post #50

Earlier quoted context omitted.

As I said, the compiler did generate FMA instructions. These are SIMD instructions, so yes, the program was vectorized.

> yes, the program was vectorized "The program" contains two hot loops. Judging from the assembly code you linked in a sibling comment, only the second of these loops was vectorized, the first one wasn't. This slower non-vectorized loop will still dominate execution time. And for whatever it's worth, dropping the original article's for i in 0..n{ b[i]=b[i]+(r/beta)*(c[i]-a[i]*b[i]) } in place of your loop using itera…

Hi I wrote the blog post linked - and I feel a little silly that I didn't check that _both_ loops vectorized. So I fixed the Rust implementation to keep a running vector of partial sums which I finish up at the end - this one did vectorize. The result was a 2X performance bump, which I'm about to include in the blog post as an update.

If it's OK I'll link to this comment as the inspiration.

On the iterators versus loop: for some reason when I use the raw loop _nothing_ vectorizes, not even the obvious loop. What I read online was that bounds checking happens inside the loop body because Rust doesn't know where those indices are coming from. Using iterators instead is supposed to fix this, and it did seem to in my experiments.

Re: Rust and C++ on Floating-Point Intensive Code

#85

Post by keldaris »

steveklabnik wrote: post #80
keldaris wrote: post #79

Earlier quoted context omitted.

I don't have anything solid backing this up, but I've often heard Rust proponents claim that the borrow checker is actually fairly cheap in terms of compile time overhead. Looking at the PR, it's obvious that it generates quite a bit of code (separate kernels for different block sizes, etc.) and does things like generics over tuples, but I don't know enough about Rust to pick out patterns that might be particularly s…

https://wiki.alopex.li/WhereRustcSpendsItsTime is a recent exploration of this topic that's very interesting.

That was interesting indeed, thanks. For an outsider, the most trivial takeaway from that is something along the lines of "Rust is slow to compile and it's probably not getting 10x faster any time soon". Beyond the obvious culprits like heavy metaprogramming, it looks like death by a thousand cuts (not to mention that ggez spends 15 seconds on something that's not even accounted for in the profile, that alone is huge) and there obviously isn't a common root cause for huge compile times in general.

Re: Rust and C++ on Floating-Point Intensive Code

#86

Post by nravic »

brandmeyer wrote: post #70

This is interesting to see. But if I'm going to compare numerical C++ against numerical Rust, then I would be using a higher-level library for the comparision. What is Rust's Other Leading Brand (TM) for the Eigen C++ library?

There's a bunch of Eigen analogues in Rust, which is slightly frustrating. ndarray is pretty great though

Re: Rust and C++ on Floating-Point Intensive Code

#87

Post by pcwalton »

diamondlovesyou wrote: post #4

I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to igno…

Sounds like it's fair to say most of this boils down to bounds checks and SIMD?

Re: Rust and C++ on Floating-Point Intensive Code

#88

Post by tom_mellior »

silentvoice wrote: post #84
tom_mellior wrote: post #67

Earlier quoted context omitted.

> yes, the program was vectorized "The program" contains two hot loops. Judging from the assembly code you linked in a sibling comment, only the second of these loops was vectorized, the first one wasn't. This slower non-vectorized loop will still dominate execution time. And for whatever it's worth, dropping the original article's for i in 0..n{ b[i]=b[i]+(r/beta)*(c[i]-a[i]*b[i]) } in place of your loop using itera…

Hi I wrote the blog post linked - and I feel a little silly that I didn't check that _both_ loops vectorized. So I fixed the Rust implementation to keep a running vector of partial sums which I finish up at the end - this one did vectorize. The result was a 2X performance bump, which I'm about to include in the blog post as an update. If it's OK I'll link to this comment as the inspiration. On the iterators versus lo…

Great to hear that you managed another 2x speedup! Sure, feel free to link my comment if you like.

Re: Rust and C++ on Floating-Point Intensive Code

#89

Post by lilyball »

nwallin wrote: post #81
paulddraper wrote: post #32

Earlier quoted context omitted.

FMA isn't a safe optimization as it can give different results. C++ compilers have flags to enable it globally. gcc and clang include the optimization in -Ofast. Rust allows you to choose at a code level (but usually people don't know about it). Perhaps it should also have a global fast-math flag that would automatically optimize it. Pros and cons to that.

FMA is "safe" in that if it breaks your code, it was already broken. It can only make the results slightly more accurate, unlike for instance the rsqrt instruction which is less accurate. (and as such is not a safe optimization) GCC emits FMA instructions at -O2 without -ffast-math.

Well, it could "break" your code in that it might make your code produce different results than a separate equivalent implementation that didn't use FMA.

Edit: Ok actually it sounds like it could literally break some algorithms, see https://news.ycombinator.com/item?id=21342974

Re: Rust and C++ on Floating-Point Intensive Code

#90

Post by jcl »

gameswithgo wrote: post #63
lovasoa wrote: post #50

Earlier quoted context omitted.

As I said, the compiler did generate FMA instructions. These are SIMD instructions, so yes, the program was vectorized.

it isn't always that simple. FMA instructions are tricky to use in a way that actually improves performance, llvm may be doing it right while doing it manually that way may not. also, sometimes a SIMD instruction is used but only on 1 lane at a time. this is actually common with floating point code.

Something I found surprising: Some AVX2 and AVX-512 instructions consume so much power that Intel chose to have their chips dynamically slow their clock frequency when the instructions are executed. So naively switching to SIMD instructions can not only fail to improve performance, but it can also hurt the performance of unaltered code executed after it -- even unrelated code running on other cores.

https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...

Post reply on HN