Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards fearless SIMD

raphlinus.github.io

Post reply on HN
Print view View original

71–80 of 82 posts

Re: Towards fearless SIMD

#71

Post by byt143 »

coder543 wrote: post #61
skemper911 wrote: post #60

In rust this looks rather painful. Might I suggest trying this in Julia, might be a good comparison of performance, ease of use, and readability. Julia does a very nice job of compiling directly to SIMD instruction and lets you inspect the low level code generated. inline function sin9_shaper(x) c0 = 6.28308759 c1 = -41.33318707 c2 = 81.39900205 c3 = -74.66884436 c4 = 33.15324345 a = abs(x - round(x)) - 0.25 a2 = a *…

Your formatting didn't work, but no, it doesn't "look painful." What you wrote does not guarantee vectorization, it just relies on autovectorization. Rust already does autovectorization magically behind the scenes thanks to LLVM (which Julia also uses), but explicit SIMD makes it a guarantee.

Here is how to do explicit SIMD in julia: https://github.com/eschnett/SIMD.jl

Re: Towards fearless SIMD

#72

Post by twtw »

tpolzer wrote: post #59
marmaduke wrote: post #44

Earlier quoted context omitted.

OpenCL on CPU is exactly this and does very well. Last real world test I did, an 8 core Haswell and a GTX 970 were similar in performance, for the kernel I was running. Caveats apply of course but it was refreshing to have a single programming model.

I did some evaluation of (Linux based) OpenCL implementations two years ago (but I don't think anything changed significantly since then). I had a few takeaways: - OpenCL on GPUs and CPUs have little to do with each other (in terms of performance characteristics) and if you tune well for one of them, the other one will suffer. - Vectorization of work items doesn't really work well unless your kernel is so simple that…

> if you tune well for one of them, the other one will suffer.

This is one of the things that bothers me the most about OpenCL. It attempts to offer this uniform abstraction over a generic compute accelerator, which can be CPU vector extensions, GPUs, or FPGAs, but these things are different enough that you have to develop for a specific type if you want reasonable performance. So you get none of the benefits of a accelerator specific abstraction while still writing accelerator specific code.

There is a real cost to a generic abstraction, and distinct languages/platforms would in my mind be better than different "dialects" of the same language/platform that pretend to be compatible but really aren't.

I like that CUDA is very clearly designed to run only on GPUs - it provides a clarity that OpenCL lacks.

Re: Towards fearless SIMD

#73

Post by a1369209993 »

ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

I think what you're looking for is ISPC.[0][1] It distinguishes between 'uniform' (scalar) vs 'varying' (vector) variables[2] and applies SIMD instructions pretty much transparently. It defaults to varying, whereas I think for a general-purpose C-like compiler you'd want scalar by default and explicitly declare eg:

  vector float add2(float base,vector float offs)
    {
    return base + 2*offs;
    }
  double sum(double* ar,size_t ln)
    {
    vector double x = (vector)0.0;
    while(ln >= sizeof vector) /* == # of simd lanes */
      {
      x += *ar; /* maybe this needs some kind of cast? */
      ln -= sizeof vector; /* == 1 on arches with no simd support */
      }
    /* this part's definitely not done properly */
    double buf[sizeof vector]; *buf = x;
    double ret = 0.0; /* slurp up the last few */
    for(size_t i=0; i
0: https://pharr.org/matt/blog/2018/04/18/ispc-origins.html

1: https://ispc.github.io/

2: https://pharr.org/matt/blog/2018/04/26/ispc-volta-more-on-pe...

Re: Towards fearless SIMD

#74

Post by davrosthedalek »

cjhanks wrote: post #67
ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

Yes, SIMD intrinsic programming is often worthless unless implemented for every target architecture. Let me give you an example case; my target production architecture rarely changes, but I want my program to run on every developer computer where CPUs are heterogeneous. Now, I think the problem is that it is tying optimizations to instructions rather than CPU architectures. When I ported code from an I7 to an I9 it s…

GCC has function multiversioning. See: https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/Function-Multiv... and since release 6 quite comfortably: https://lwn.net/Articles/691932/

Re: Towards fearless SIMD

#75

Post by davrosthedalek »

Rebelgecko wrote: post #19
blattimwind wrote: post #10

Earlier quoted context omitted.

> In an LLVM context it also means you don't get runtime feature detection. You would need to build N dlls and then write code to load the proper one at runtime based on feature detection. Does LLVM not support GCC-style function multiversioning?

Does GCC use that when auto-vectorizing? It's been a while, but in the past when I built binaries via gcc with autogenerated SSE/AVX I don't think they fell back to the C code for CPUs that lacked those SIMD instructions. They just crashed.

You have to tell gcc it should also build a generic version. Then, it should work.

Re: Towards fearless SIMD

#76

Post by marmaduke »

tpolzer wrote: post #59
marmaduke wrote: post #44

Earlier quoted context omitted.

OpenCL on CPU is exactly this and does very well. Last real world test I did, an 8 core Haswell and a GTX 970 were similar in performance, for the kernel I was running. Caveats apply of course but it was refreshing to have a single programming model.

I did some evaluation of (Linux based) OpenCL implementations two years ago (but I don't think anything changed significantly since then). I had a few takeaways: - OpenCL on GPUs and CPUs have little to do with each other (in terms of performance characteristics) and if you tune well for one of them, the other one will suffer. - Vectorization of work items doesn't really work well unless your kernel is so simple that…

The Intel OpenCL driver vectorized work items really well, better than the same code provided to ICC as C. It was still more fragile than CUDA.

The other points are spot on and I would add that debugging code in OpenCL is a bad experience.

Re: Towards fearless SIMD

#77

Post by user_1234 »

> The next-level challenge is compiling multiple alternates into the same binary, and selecting the best at runtime.

What are the use cases for having multiple alternates in the same binary? Why not decide them during the compile time for the given architecture?

If portability is the concern here, wouldn't it lead to sub-optimal code anyway?

Re: Towards fearless SIMD

#78

Post by pjmlp »

jandrewrogers wrote: post #52
ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

I find intrinsics to be a useful level of abstraction because it allows me to control and automate the code generation in C++ without having to know the details of register scheduling, etc. It also means I don't have to hand write intrinsic algorithms for every architecture -- I have template libraries that can take a rough specification of the architecture characteristics and generate approximately appropriate intri…

As far as I know from reading papers, Fortran compilers do a much better job thanks to the language semantics.

Re: Towards fearless SIMD

#79

Post by rcxdude »

user_1234 wrote: post #77

> The next-level challenge is compiling multiple alternates into the same binary, and selecting the best at runtime. What are the use cases for having multiple alternates in the same binary? Why not decide them during the compile time for the given architecture? If portability is the concern here, wouldn't it lead to sub-optimal code anyway?

If you're shipping binaries, you don't know the exact architecture in advance (because there are many extensions to x86 and you don't know if the end user is running a new enough processor to use all of them). If you don't use them you are likely leaving performance on the table. So you want to select the fastest option supported by the processor you happen to be running on. You can do this with fairly minimal peformance impact by linking in different versions of the function at runtime, but this requires some support from your compiler and runtime environment.

Re: Towards fearless SIMD

#80

Post by cjhanks »

davrosthedalek wrote: post #74
cjhanks wrote: post #67

Earlier quoted context omitted.

Yes, SIMD intrinsic programming is often worthless unless implemented for every target architecture. Let me give you an example case; my target production architecture rarely changes, but I want my program to run on every developer computer where CPUs are heterogeneous. Now, I think the problem is that it is tying optimizations to instructions rather than CPU architectures. When I ported code from an I7 to an I9 it s…

GCC has function multiversioning. See: https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/Function-Multiv... and since release 6 quite comfortably: https://lwn.net/Articles/691932/

Thank you so much for sharing this, I did not know. I guess I will have to look for the version GCC version which supported X86_64.. I might have to upgrade distros for GCC6 (older versions were i386).

Btw, this was a nice presentation of the idea: https://blog.linuxplumbersconf.org/2016/ocw/system/presentat...

Post reply on HN