Everyone should know SIMD • Hacker News Board

Re: Everyone should know SIMD

#131

Post by exDM69 » Thu, Jul 23, 2026, 7:37 AM UTC

losvedir wrote: post #44

This is an interesting article. I don't really work with low level enough languages for this to matter (unless - does this ever show up in Javascript somehow?). I guess I don't understand the "reduce" step. It seems like you have to be careful not to "undo" all the benefit from SIMD. Sure, it can compare 8 values in parallel, but then if you have to look at each of the 8 answers in turn you're back to where you began…

> Is the `@reduce()` function in the example a special...

Yes, it is. In SIMD you can do a reduce for a commutative function in O(log N) steps. Sum, product, min, max, all, any, etc.

Although in this case it might be better if you just take the mask (vector of booleans), convert it to a bitmask and check if it is (non-) zero.

If the language provides it, you might also use .all() or .any() for a mask.

Re: Everyone should know SIMD

#132

Post by avaer » Thu, Jul 23, 2026, 7:40 AM UTC

I agree in spirit, but most serious cases of this class of problem have moved to accelerated kernels (e.g. GPU). Which has some commonality but is different enough that a lot of these learnings don't translate.

There might be a narrow class of problem where:

  - SISD is the bottleneck
  - Compiler won't autovectorize
  - Data is small or weird enough, or the environment constrained enough that running it on an accelerator is not feasible

But that seems an increasingly small scope for something "everyone should know".

Re: Everyone should know SIMD

#133

Post by exDM69 » Thu, Jul 23, 2026, 7:42 AM UTC

forrestthewoods wrote: post #3

qurren wrote: post #2

I just do gcc -O3 and get SIMD without having to learn it

auto-vectorization is not nearly as good as you would hope it to be. The best SIMD optimizations likely require changing your data format from AoS to SoA.

> The best SIMD optimizations likely require changing your data format from AoS to SoA.

We do have gather load instructions in SIMD instruction sets these days (AVX2 and newer), so AoS vs SoA is not nearly as important as it was once.

Scatter stores are also available but only in newer CPUs.

Re: Everyone should know SIMD

#134

Post by menaerus » Thu, Jul 23, 2026, 7:45 AM UTC

boricj wrote: post #46

Everyone doesn't need to know SIMD. Mechanical sympathy is an important passive perk for software architects to cut down the number of reworks down the line, but I would rate benchmarking and being able to identify bottlenecks as more important everyday skills. I'm working on a voxel space renderer homebrew for the PlayStation. I only have so many cycles to spend on rendering before it becomes a slideshow, so I count…

To do some optimization work with SIMD what you actually need to understand is the underlying CPU uarchitecture, intrinsics by the end of the day are just an API. But to also make sense of the benchmarking results or bottleneck debugging you also need to understand the underlying CPU uarchitecture and/or further devices your workload might be utilizing, e.g. storage.

Re: Everyone should know SIMD

#135

Post by hashmal » Thu, Jul 23, 2026, 7:45 AM UTC

avaer wrote: post #132

I agree in spirit, but most serious cases of this class of problem have moved to accelerated kernels (e.g. GPU). Which has some commonality but is different enough that a lot of these learnings don't translate. There might be a narrow class of problem where: - SISD is the bottleneck - Compiler won't autovectorize - Data is small or weird enough, or the environment constrained enough that running it on an accelerator…

or when you have stronger real-time constraints than "send it to the GPU and hope it comes back fast enough" (e.g. real-time audio processing)

Re: Everyone should know SIMD

#136

Post by menaerus » Thu, Jul 23, 2026, 7:48 AM UTC

saagarjha wrote: post #97

guess_who_is wrote: post #55

SIMD does not pay, speaking from 20yr exp in the field in various semis.

It pays quite well if you know who to work for

I think what the parent comment sentiment was that having this knowledge generally does not pay off but there are certainly positions which do ask for it specifically. And they are very few IME.

Re: Everyone should know SIMD

#137

Post by imtringued » Thu, Jul 23, 2026, 7:51 AM UTC

Joker_vD wrote: post #115

alberth wrote: post #99

Earlier quoted context omitted.

That’s exactly what happened: https://xcancel.com/mitchellh/status/2079672171321081908#m

You know, it's always funny to read takes like "A broken compiler forcing you to write explicit SIMD instead of trusting auto-vectorization and coming out 20-30% faster is the best argument I've seen for reading your own generated assembly occasionally instead of assuming the compiler has you covered" because you can quite easily imagine an alternative one like "A broken compiler revealing that the auto-vectorization…

Yeah it's strange how they brushed away that the compiler reached 77% of the hand crafted performance without even trying.

Re: Everyone should know SIMD

#138

Post by inigyou » Thu, Jul 23, 2026, 7:57 AM UTC

crabmusket wrote: post #117

hnal943 wrote: post #17

Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqY

It's a great talk, I just wish there was a good focused textual version of it, as it is a very long video to recommend to others. Very worth it, but a big investment. It's a great example of what I think of as vertical integration for performance. As you go through the talk you can understand why all these abstractions exist and why they have to be so generic. But when you have a specific use case, you can vertically…

Be the change you want to see. Post a transcript on your own website.

Re: Everyone should know SIMD

#139

Post by t-3 » Thu, Jul 23, 2026, 7:59 AM UTC

teo_zero wrote: post #113

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

SIMD is simple, using data-parallel operations in scalar languages is what's awkward.

Re: Everyone should know SIMD

#140

Post by inigyou » Thu, Jul 23, 2026, 7:59 AM UTC

tstack wrote: post #66

derf_ wrote: post #29

To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible. Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the…

> what can be fast I think this doesn't get talked about enough. If your input is a big run of data that is being checked/transformed in one shot, it works well. But, if you're likely to have to make a decision on several bytes of the input, SIMD will be the same or slower than the scalar method. It's not a magic "go fast" button.

That is not necessarily true. simdjson exists. But it's far from simple.

I have been told SIMD is good for data-parallel loops, like the GPU is, and that is true, but it can also be used piecemeal, unlike the GPU. Because it is just the CPU, you can read 32 unaligned bytes, scan for the index of the first space, and take a branch based on that.