Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards fearless SIMD

raphlinus.github.io

Post reply on HN
Print view View original

31–40 of 82 posts

Re: Towards fearless SIMD

#31

Post by gameswithgo »

Bjartr wrote: post #12
gameswithgo wrote: post #7

Another Rust project with similar goals, different approach: https://github.com/jackmott/simdeez full disclosure, that's mine.

Your library is actually mentioned in the linked article > Another crate, with considerable overlap in goals, is simdeez. This crate is designed to facilitate runtime detection, and writing the actual logic without duplication, but leaves the actual writing of architecture specific shims to the user, and still requires nontrivial unsafe code.

cool, he updated it, it wasn't originally. We have chatted a bit and exchanged ideas.

Re: Towards fearless SIMD

#32

Post by pcwalton »

littlestymaar wrote: post #26
pcwalton wrote: post #11

Earlier quoted context omitted.

Nice! This is much simpler than faster. Thanks for dropping the link here. I really need to revive my multithreaded, SIMD-ified Rust PNG decoder someday with something like this…

Weren't your working on a PNG decoder running on the GPU ?

Yeah, I was, but branch divergence made it slower than just using SIMD.

Re: Towards fearless SIMD

#33

Post by gameswithgo »

pjmlp wrote: post #13
gameswithgo wrote: post #8

Earlier quoted context omitted.

In an LLVM context it also means you don't get runtime feature detection. You would need to build N dlls and then write code to load the proper one at runtime based on feature detection. JITs could solve that problem, but few JITs currently do very much auto vectorization, because they don't have time.

Intel has made quite a few contributions to Hotspot, including AVX support. ART started supporting SIMD on Oreo, and it was further expanded on Pie. Naturally very few devices have gotten those improvements thanks the state of Android's updates. So on Android's case Renderscript is still the best way for a JIT like approach for SIMD.

As far as I know the JVM will only auto-vectorize integers. Is there a way to tag a function such that you want floats vectorized too now?

ART is a good point, compile on install lets you do this.

Re: Towards fearless SIMD

#34

Post by pcwalton »

ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

The compiler can optimize intrinsics, though. Why shouldn't it be able to?

I agree that somewhat with AVX2 and especially with AVX-512 there's probably less reason to write intrinsics if you have a vector language. But for now SSE2, SSSE3, and SSE4 are still the bread and butter for SIMD on x86, and there are some important instructions (Fabien Giesen goes into detail at [1]) that you really have to think about how to use effectively. For example (this is mentioned at the end), all horizontal adds on x86 are awful except for PSADBW, which is really limited, as Intel designed it in a fit of myopia to target only motion estimation in contemporary codecs, and it requires you to basically design your whole algorithm around it.

[1]: https://fgiesen.wordpress.com/2016/04/03/sse-mind-the-gap/

Re: Towards fearless SIMD

#35

Post by burntsushi »

amluto wrote: post #23

Why is SIMD considered unsafe? I thought that safe code was permitted to cause panics, and the worst thing that will happen if unsupported SIMD is used is a panic.

No, it's not about panicking. It's undefined behavior to run code compiled with CPU features that aren't supported by the current CPU. See: https://github.com/rust-lang/rfcs/blob/master/text/2045-targ...

There are some other ideas for making it easier to reason about safety at this level: https://github.com/rust-lang/rfcs/pull/2212

Can you point to where you heard about unsupported SIMD causing a panic? I'd like to fix that because it's really wrong!

Re: Towards fearless SIMD

#36

Post by grandinj »

stochastic_monk wrote: post #14

I’ve found template metaprogramming to be an efficient, generic way to generate SIMD-accelerated code. (See [0].) By giving the type functions associated with each operation, the same interface can support 128, 256, or 512-bit SIMD, depending on hardware. It’s C++, so while it’s outside of the Rust ecosystem, it’s still a workable solution I’ve used in a half dozen projects. [0] https://github.com/dnbaker/vec

That doesn't give you run-time dispatch based on available hardware capabilities. So you either need virtual functions (and take the associated perf hit), or some other solution like the linker tricks mentions, or cached JIT, or....

Re: Towards fearless SIMD

#37

Post by stochastic_monk »

grandinj wrote: post #36
stochastic_monk wrote: post #14

I’ve found template metaprogramming to be an efficient, generic way to generate SIMD-accelerated code. (See [0].) By giving the type functions associated with each operation, the same interface can support 128, 256, or 512-bit SIMD, depending on hardware. It’s C++, so while it’s outside of the Rust ecosystem, it’s still a workable solution I’ve used in a half dozen projects. [0] https://github.com/dnbaker/vec

That doesn't give you run-time dispatch based on available hardware capabilities. So you either need virtual functions (and take the associated perf hit), or some other solution like the linker tricks mentions, or cached JIT, or....

It’s not runtime, it’s compile-time. That means it needs to be separately compiled for each hardware backend.

Re: Towards fearless SIMD

#38

Post by jcranmer »

ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

For the simple SIMD cases where the computation doesn't have any horizontal dependencies (except for a total reduction step), there's little need for any sort of intrinsic usage. (Although you may need some compiler hints to push it to autovectorize).

The real problem is that SIMD hardware sets tends to have lots of instructions that have mixing of horizontal lanes. The scope and performance of these mixing instructions varies greatly from implementation to implementation, and these kinds of instructions are hard for compilers to automatically pick up. It's the latter case that means you need the SIMD intrinsics to be exposed to use the hardware effectively.

Re: Towards fearless SIMD

#39

Post by grandinj »

stochastic_monk wrote: post #37
grandinj wrote: post #36

Earlier quoted context omitted.

That doesn't give you run-time dispatch based on available hardware capabilities. So you either need virtual functions (and take the associated perf hit), or some other solution like the linker tricks mentions, or cached JIT, or....

It’s not runtime, it’s compile-time. That means it needs to be separately compiled for each hardware backend.

How do you combine the separately compiled libraries into a single deliverable library/executable?

Because that is what the original article is aiming for.

Re: Towards fearless SIMD

#40

Post by stochastic_monk »

grandinj wrote: post #39
stochastic_monk wrote: post #37

Earlier quoted context omitted.

It’s not runtime, it’s compile-time. That means it needs to be separately compiled for each hardware backend.

How do you combine the separately compiled libraries into a single deliverable library/executable? Because that is what the original article is aiming for.

It doesn’t do the runtime dispatch this person wants, but it eliminates the worry of running into an illegal instruction error.

I think there are use cases for a fat binary, but I’m less certain as to its necessity.

Post reply on HN