Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards fearless SIMD

raphlinus.github.io

Post reply on HN
Print view View original

61–70 of 82 posts

Re: Towards fearless SIMD

#61

Post by coder543 »

skemper911 wrote: post #60

In rust this looks rather painful. Might I suggest trying this in Julia, might be a good comparison of performance, ease of use, and readability. Julia does a very nice job of compiling directly to SIMD instruction and lets you inspect the low level code generated. inline function sin9_shaper(x) c0 = 6.28308759 c1 = -41.33318707 c2 = 81.39900205 c3 = -74.66884436 c4 = 33.15324345 a = abs(x - round(x)) - 0.25 a2 = a *…

Your formatting didn't work, but no, it doesn't "look painful."

What you wrote does not guarantee vectorization, it just relies on autovectorization.

Rust already does autovectorization magically behind the scenes thanks to LLVM (which Julia also uses), but explicit SIMD makes it a guarantee.

Re: Towards fearless SIMD

#62

Post by joe_the_user »

zackmorris wrote: post #29
ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

I think so too, that SIMD is too low-level to be utilized effectively. Luckily joe_the_user enlightened me with this little gem on a previous post: https://news.ycombinator.com/item?id=17419917 Or for the lazy: I don't know about tensor flow in particular but are little-known methods of running "general purpose" parallel programs on GPUs. Specifically, H. Dietz' MOG, "Mimd on GPU". It's a shame the project hasn't got…

Cool!

I have "evangelized" one person at least.

The thing about MOG is it's a demonstrated technique that Dietz has not yet developed into a fully releasable product. His latest comment says it's six months from release but lacks funding.

I think the problem might be that most "Mimd" programming is things like weather-simulations where access to a MIMD supercomputer is standard and price isn't that much of an issue.

That said, cheap Simd parallel computing jump-started today's deep learning advances and so cheap MIMD might do things no one anticipated.

Still, very niche. But thanks for the mention.

Edit: Note, MOG is specifically a GPU. Dietz did earlier work on other machines in the 90s but this is GPU specific (though a less flexible architecture than what Nvidia has evolved now).

Re: Towards fearless SIMD

#63

Post by ndesaulniers »

jandrewrogers wrote: post #52
ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

I find intrinsics to be a useful level of abstraction because it allows me to control and automate the code generation in C++ without having to know the details of register scheduling, etc. It also means I don't have to hand write intrinsic algorithms for every architecture -- I have template libraries that can take a rough specification of the architecture characteristics and generate approximately appropriate intri…

> All of the "higher level abstractions" I've seen are no better at finding vectorizable patterns than the compiler does

I think a lot of that comes from C/C++ language corners.

Have you seen XLA (https://www.tensorflow.org/performance/xla/) or Glow (https://facebook.ai/developers/tools/glow)? These are very high level abstractions called from C++ that can do a great job vectorizing high level operations for the given device.

Re: Towards fearless SIMD

#64

Post by raphlinus »

skemper911 wrote: post #60

In rust this looks rather painful. Might I suggest trying this in Julia, might be a good comparison of performance, ease of use, and readability. Julia does a very nice job of compiling directly to SIMD instruction and lets you inspect the low level code generated. inline function sin9_shaper(x) c0 = 6.28308759 c1 = -41.33318707 c2 = 81.39900205 c3 = -74.66884436 c4 = 33.15324345 a = abs(x - round(x)) - 0.25 a2 = a *…

From the asm you posted I have no idea whether it's optimized or not; it looks like two virtual function calls (calll %eax). Also, while I'm sure Julia is nice, the GC makes me worry about whether it'll work well in real-time audio synthesis (my primary use case).

Re: Towards fearless SIMD

#66

Post by shaklee3 »

zackmorris wrote: post #58
gameswithgo wrote: post #53

Earlier quoted context omitted.

what do you mean by effectively? Too hard for the programmer? Because what most of us find when we try to do SIMD stuff is anything higher level than intrinsics is too hard to use effectively where effective means good runtime performance. I agree there should be something higher level we could use but it doesn't exist yet, to my knowledge.

Ya too hard for the programmer. What it comes down to is that there are a few main categories of multiprocessing from lowest to highest level: 1. SIMD - manually deal with packed elements (like in SSE, MMX etc) 2. DSP - Maybe someone knows a better term for this, but treating each slice of the data array as an independent serial stream (shaders, OpenCL, CUDA) 3. MIMD - freeform vector/matrix operations that get compi…

The eigen c++ library comes somewhat close. But remember, Matlab is terrible at using multiple processors, so most people using Matlab won't see anywhere the performance most people c intrinsics get. This isn't because Matlab isn't capable, but I found that virtually no Matlab users try to write parallel code.

Re: Towards fearless SIMD

#67

Post by cjhanks »

ndesaulniers wrote: post #16

> Code written for too high a SIMD capability will generally crash (it’s undefined behavior), while code written for too low a SIMD capability will fall short of the performance available. I can't help but think that SIMD intrinsics are the wrong level of abstraction. Literally assembly being moved into a high level language but without the ability for the compiler to optimize well. Hand written intrinsics can really…

Yes, SIMD intrinsic programming is often worthless unless implemented for every target architecture. Let me give you an example case; my target production architecture rarely changes, but I want my program to run on every developer computer where CPUs are heterogeneous.

Now, I think the problem is that it is tying optimizations to instructions rather than CPU architectures. When I ported code from an I7 to an I9 it seemed that instruction latency changed. Still, if you are only supporting limited computational targets any hooks at all help.

But why intrinsic rather than assembly? Because usually vector extensions in GCC work well enough, and when they don't intrinsics are easier for me (a man who rarely needs to write assembly and doesn't have the time to beat the compiler in 99.99% of cases).

And a minor point... I really wish GCC had a pragma which allowed me to specify a function should be optimized to multiple targets and then install the thunks automatically.

Re: Towards fearless SIMD

#68

Post by pbsd »

amluto wrote: post #55
burntsushi wrote: post #35

Earlier quoted context omitted.

No, it's not about panicking. It's undefined behavior to run code compiled with CPU features that aren't supported by the current CPU. See: https://github.com/rust-lang/rfcs/blob/master/text/2045-targ... There are some other ideas for making it easier to reason about safety at this level: https://github.com/rust-lang/rfcs/pull/2212 Can you point to where you heard about unsupported SIMD causing a panic? I'd like to f…

I mean that it really shouldn’t be UB to call a function compiled for an unsupported target feature. Following that link, I see two arguments that it’s UB: 1. A multibyte NOP might be used. Supposedly there might be a multibyte NOP that older CPUs will decode as a jump. I am not sure I believe this. Is there an example? 2. int3 might happen, causing SIGTRAP. I see no explanation of how this would occur. So I think th…

There is the case of LZCNT and BSR. On processors that do not support the former, LZCNT is interpreted as (REP) BSR, but the instructions have different behavior that could result in silent failures.

Re: Towards fearless SIMD

#69

Post by davidhyde »

> Note that this is a performance of approximately 470 picoseconds per sample. Modern computers are fast when running optimized code.

Or about half the time it takes a photon from your phone screen to hit your eyeball

Re: Towards fearless SIMD

#70

Post by kristofferc »

skemper911 wrote: post #60

In rust this looks rather painful. Might I suggest trying this in Julia, might be a good comparison of performance, ease of use, and readability. Julia does a very nice job of compiling directly to SIMD instruction and lets you inspect the low level code generated. inline function sin9_shaper(x) c0 = 6.28308759 c1 = -41.33318707 c2 = 81.39900205 c3 = -74.66884436 c4 = 33.15324345 a = abs(x - round(x)) - 0.25 a2 = a *…

Your code only shows that

    gen_sinwave(113.0)
calls

    gen_sinwave(1113.0, 0.0, 0.1)
(which are the default arguments). It is usually better to use @code_llvm because the LLVM IR is typically easier to read.

    julia> @code_llvm gen_sinwave(1113.0);
    
    ; Function gen_sinwave
    ; Location: REPL[2]:2
    define nonnull %jl_value_t addrspace(10)* 
    @julia_gen_sinwave_345638059(double) {
    top:
      %1 = call nonnull %jl_value_t addrspace(10)* @julia_gen_sinwave_345638060(double %0, double 0.000000e+00, double 1.000000e-01)
      ret %jl_value_t addrspace(10)* %1
    }
However, defining

    function gen_sinwave(freq, init=0.0, step=0.1)
        data = collect(init:step:freq)
        fin9_shaper.(data)
    end
and looking at

    @code_llvm gen_sinwave(1113.0, 0.0, 1.0)
we can see that there is a lot of auto-vectorization going on
Post reply on HN