Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards fearless SIMD

raphlinus.github.io

Post reply on HN
Print view View original

21–30 of 82 posts

Re: Towards fearless SIMD

#21

Post by raphlinus »

wyldfire wrote: post #6

The simplest target-independent fearless SIMD is autovectorization[1] . But taking maximum advantage of that probably means writing some code that feels a little unnatural. Also, IIRC bounds checks thwart some autovectorization. [1] https://llvm.org/docs/Vectorizers.html

I've spent quite a bit of time looking at autovectorization, but didn't write about it much here, as it's only good for a pretty small subset of problems. One subtle gotcha I ran into is that `round` doesn't autovectorize, but `float` does, even though today's chips have perfectly good vector round instructions. See rust issue #55107 for a deep dive into that problem.

Re: Towards fearless SIMD

#22

Post by eslaught »

stochastic_monk wrote: post #14

I’ve found template metaprogramming to be an efficient, generic way to generate SIMD-accelerated code. (See [0].) By giving the type functions associated with each operation, the same interface can support 128, 256, or 512-bit SIMD, depending on hardware. It’s C++, so while it’s outside of the Rust ecosystem, it’s still a workable solution I’ve used in a half dozen projects. [0] https://github.com/dnbaker/vec

I don't know if you've ever seen Kokkos[1], but that's one of the big C++ frameworks for taking parallel loops and using templates to generate code for SIMD and multiple cores. They also support GPU code generation to some degree. The main issue is compile time, which gets pretty bad when you start using it in non-toy applications.

I can't find the reference right now, but there have been some attempts to augment C++ compilers to understand the semantics of the library directly, basically treating the template metaprogramming as a DSL instead of general-purpose. This can be speed up the compile times dramatically, but of course you lose generality and the compiler has to be customized to understand every library it wants to optimize. Overall it seems like there is more research to be done on doing this in a safe way without paying through the nose with compile time.

[1]: https://github.com/kokkos/kokkos

Re: Towards fearless SIMD

#23

Post by amluto »

Why is SIMD considered unsafe? I thought that safe code was permitted to cause panics, and the worst thing that will happen if unsupported SIMD is used is a panic.

Re: Towards fearless SIMD

#24

Post by raphlinus »

blattimwind wrote: post #10
gameswithgo wrote: post #8

Earlier quoted context omitted.

In an LLVM context it also means you don't get runtime feature detection. You would need to build N dlls and then write code to load the proper one at runtime based on feature detection. JITs could solve that problem, but few JITs currently do very much auto vectorization, because they don't have time.

> In an LLVM context it also means you don't get runtime feature detection. You would need to build N dlls and then write code to load the proper one at runtime based on feature detection. Does LLVM not support GCC-style function multiversioning?

LLVM does (quite recently!), but Rust currently does not. I think it's likely that Rust should add it, but a large part of my post was exploring how far we could go with Rust as of today.

Edit adding citation: http://lists.llvm.org/pipermail/llvm-announce/2018-September...

Re: Towards fearless SIMD

#25

Post by Const-me »

nikic wrote: post #17
ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

Half the point of using SIMD intrinsics rather than embedding assembly is that register allocation and instruction scheduling are performed by the compiler. Using intrinsics certainly does not guarantee that exactly the corresponding instructions will be emitted in exactly the given order.

> Using intrinsics certainly does not guarantee that exactly the corresponding instructions will be emitted in exactly the given order.

I hope they will fix the compilers to add such guarantees. Currently, every time compilers try to mess with manually-written intrinsics, they decrease performance.

See this bug about LLVM failing to emit the exact instructions, decreasing the performance: https://bugs.llvm.org/show_bug.cgi?id=26491

See this bug about VC++ failing to emit them in the given order, again decreasing performance: https://developercommunity.visualstudio.com/content/problem/...

Re: Towards fearless SIMD

#26

Post by littlestymaar »

pcwalton wrote: post #11
gameswithgo wrote: post #7

Another Rust project with similar goals, different approach: https://github.com/jackmott/simdeez full disclosure, that's mine.

Nice! This is much simpler than faster. Thanks for dropping the link here. I really need to revive my multithreaded, SIMD-ified Rust PNG decoder someday with something like this…

Weren't your working on a PNG decoder running on the GPU ?

Re: Towards fearless SIMD

#27

Post by nikic »

Const-me wrote: post #25
nikic wrote: post #17

Earlier quoted context omitted.

Half the point of using SIMD intrinsics rather than embedding assembly is that register allocation and instruction scheduling are performed by the compiler. Using intrinsics certainly does not guarantee that exactly the corresponding instructions will be emitted in exactly the given order.

> Using intrinsics certainly does not guarantee that exactly the corresponding instructions will be emitted in exactly the given order. I hope they will fix the compilers to add such guarantees. Currently, every time compilers try to mess with manually-written intrinsics, they decrease performance. See this bug about LLVM failing to emit the exact instructions, decreasing the performance: https://bugs.llvm.org/show_b…

As usual, things aren't quite that simple. While it certainly can happen that the compiler can pessimize carefully crafted SIMD code by applying undesirable transformations, simply not optimizing intrinsics would lead to other problems.

If you consider SIMD code that spans more than just one small function, then it is quite useful if the compiler can SROA (e.g. you pass around an array of vectors instead of having 8 arguments to each function, but still want them to end up in registers eventually), LICM (you're calling a function that needs a constant vector in a loop) or otherwise optimize your code.

If you want the compiler to leave alone your intrinsics, link in an assembly file.

Re: Towards fearless SIMD

#29

Post by zackmorris »

ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

I think so too, that SIMD is too low-level to be utilized effectively. Luckily joe_the_user enlightened me with this little gem on a previous post: https://news.ycombinator.com/item?id=17419917

Or for the lazy:

I don't know about tensor flow in particular but are little-known methods of running "general purpose" parallel programs on GPUs. Specifically, H. Dietz' MOG, "Mimd on GPU". It's a shame the project hasn't gotten more attention imo.

http://aggregate.org/MOG/

See: https://en.wikipedia.org/wiki/Flynn%27s_taxonomy for explanations of terms.

Re: Towards fearless SIMD

#30

Post by Const-me »

nikic wrote: post #27
Const-me wrote: post #25

Earlier quoted context omitted.

> Using intrinsics certainly does not guarantee that exactly the corresponding instructions will be emitted in exactly the given order. I hope they will fix the compilers to add such guarantees. Currently, every time compilers try to mess with manually-written intrinsics, they decrease performance. See this bug about LLVM failing to emit the exact instructions, decreasing the performance: https://bugs.llvm.org/show_b…

As usual, things aren't quite that simple. While it certainly can happen that the compiler can pessimize carefully crafted SIMD code by applying undesirable transformations, simply not optimizing intrinsics would lead to other problems. If you consider SIMD code that spans more than just one small function, then it is quite useful if the compiler can SROA (e.g. you pass around an array of vectors instead of having 8…

> it is quite useful if the compiler can SROA

__attribute__((always_inline)) / __forceinline usually help.

> LICM (you're calling a function that needs a constant vector in a loop)

I can calculate that vector outside of the loop.

> link in an assembly file.

Way more complex, I need to write outer scalar code in assembly too, need to manually allocate registers, also C++ templates are sometimes very useful for that kind of code.

In 99% of cases intrinsics are good enough for me, but I would love the compilers to leave alone my intrinsics.

Post reply on HN