Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should know SIMD

mitchellh.com

Post reply on HN
Print view View original

211–220 of 263 posts

Re: Everyone should know SIMD

#211

Post by spider-mario »

Joker_vD wrote: post #115
alberth wrote: post #99

Earlier quoted context omitted.

That’s exactly what happened: https://xcancel.com/mitchellh/status/2079672171321081908#m

You know, it's always funny to read takes like "A broken compiler forcing you to write explicit SIMD instead of trusting auto-vectorization and coming out 20-30% faster is the best argument I've seen for reading your own generated assembly occasionally instead of assuming the compiler has you covered" because you can quite easily imagine an alternative one like "A broken compiler revealing that the auto-vectorization…

Explicit SIMD doesn’t have to mean hand-crafting assembly (and I don’t think it did in this instance).

Re: Everyone should know SIMD

#212

Post by dotancohen »

Someone wrote: post #182
teo_zero wrote: post #113

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

>> writing SIMD is just about as easy as a for loop > and then the first example requires 12 lines to replace one line of scalar code. > Be honest and say SIMD is hard but the results are worth it! I think SIMD, and certainly that first example is way more tedious than it is hard . What makes it tedious are - you have to figure out how many things your hardware can do in parallel - you have to chop up the work in pac…

I hate to be the one to invoke AI in this otherwise virgin thread, but AI is absolutely terrific for doing the not-hard, tedious work. This seems like a place where AI tools could be used to help the user learn - so long as he instructs the tool for each small task and doesn't lazily just have the tool do the thinking for him.

Re: Everyone should know SIMD

#213

Post by randusername »

kiaansaraiya wrote: post #62

I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.

> Learning to check the compliers optimization reports is arguably more valuable

Where do I start?

I want to trust the compiler, but I don't always have time to feed every little piece into compiler explorer and interpret it. Is there a higher-level workflow?

Re: Everyone should know SIMD

#214

Post by corysama »

dsauerbrun wrote: post #170

And here I am pooping out simple typescript... These articles always make me feel like I'm wasting my talents working on products that don't really have the need to leverage any understanding of what's happening under the hood at the CPU instructions level.

Writing a Gameboy emulator is a great cure for that ;)

Re: Everyone should know SIMD

#215

Post by jmull »

Here's some important accompanying advice:

* If you aren't profiling, don't bother

Otherwise, you are going to be wasting plenty of time on "optimizations" that don't actually do anything useful.

Just come up with at least a few test that represent important cases and time them. Dig in, see where the time is being spent, and focus on areas where significant time is spent, especially ones that look ripe for optimization.

Also:

* Think about laying out your data in ways that accommodate your access patterns and are cache-friendly. SIMD would typically follow from this, not be a starting point on its own.

Re: Everyone should know SIMD

#216

Post by dnautics »

fancyfredbot wrote: post #22
dnautics wrote: post #21

Earlier quoted context omitted.

> some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector). this is reasonable because there isn't really a generalizable "good way" to unroll trig functions for simd. if you really care about speed youre better off implementing to the…

I bet there's a better way than unpacking, running sequentially and repacking. Even if the algorithm is very branchy you save a pack and unpack.

"better" - for whom? will it cause problems if, for example you are writing scientific code?

Re: Everyone should know SIMD

#217

Post by paulddraper »

Sesse__ wrote: post #187
heja2009 wrote: post #168

Earlier quoted context omitted.

Around 1990 I had the fortune to learn Parallel-C - a language that was designed during the Transputer hype and was essentially C extended by a few features to support easy parallel programming. My favourite feature was par( ; ; ) essentially a for loop that will be auto-parallelized by the compiler - some boundary conditions apply.

You may be interested in OpenMP.

OpenMP is interesting, though uses threads.

Re: Everyone should know SIMD

#218

Post by relssiegp »

teo_zero wrote: post #113

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

I had the same thoughts about SIMD code being too verbose when I wrote some, so a few months ago I tried writing a library that lets you write quasi-GLSL code in C++, so much more compact, with the ability to switch between SIMD width without having to rewrite anything at all:

https://github.com/gitdepierre/cppshader

Not sure it will ever be useful, but it was a fun pet project with some interesting problems to solve.

Re: Everyone should know SIMD

#219

Post by spider-mario »

lz400 wrote: post #84

Isn't the better abstraction here to use a higher level library in the style of pandas/polars that will operate as vectors, compose and feel readable and inuitive, while (almost?) maxing out SIMD?

That can easily cause you to traverse your data several times when once would be enough. Let’s say you wanted to compute “mean of array divided by max in absolute value”.

NumPy-like:

    mean = np.mean(array)
    maximum = np.max(np.abs(array)
    return mean / maximum
As far as I’m aware, that will be evaluated as three traversals.

Highway:

    HWY_FULL(float) d;
    using V = decltype(hn::Zero(d));

    V sum = hn::Zero(d);
    V max = hn::Zero(d);
    hn::Foreach(d, values, N, hn::Zero(d), [&](auto d, auto v) HWY_ATTR {
      sum = hn::Add(sum, v);
      max = hn::Max(max, hn::Abs(v));
    });
    const float mean = hn::ReduceSum(d, sum) / N;
    return mean / hn::ReduceMax(d, max);
Just one pass.

When the actual computation is made so much faster by SIMD, memory bandwidth starts to be a significant bottleneck.

Re: Everyone should know SIMD

#220

Post by puilp0502 »

relssiegp wrote: post #218
teo_zero wrote: post #113

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

I had the same thoughts about SIMD code being too verbose when I wrote some, so a few months ago I tried writing a library that lets you write quasi-GLSL code in C++, so much more compact, with the ability to switch between SIMD width without having to rewrite anything at all: https://github.com/gitdepierre/cppshader Not sure it will ever be useful, but it was a fun pet project with some interesting problems to solve…

Is GLSL more "approachable" than SIMD? For me personally (who doesn't have any graphics programming experience), GLSL feels way scarier than SIMD, especially when you look at the black magic that happens on shadertoys. Not saying GLSL is actually hard, but for a programmer like me SIMD might actually be more approachable
Post reply on HN