Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should know SIMD

mitchellh.com

Post reply on HN
Print view View original

231–240 of 263 posts

Re: Everyone should know SIMD

#231

Post by Archit3ch »

sgt wrote: post #167

Is SIMD in rust still pretty bad?

In some cases the ergonomics are worse than C, which is a feat in 2026.

Portable SIMD is (perma?) nightly.

Requires 'unsafe' everywhere.

To skip bounds-checking, you typically need to switch your writing style from loops to iterators.

No JIT, so you need multiversioning and/or target-cpu=native. Since Rust doesn't bring a compiler, you need to predict all your target architectures in advance.

Cannot inspect @code_llvm/@code_native at the function level like Julia, you need to compile the entire app.

Prioritizes Floating-Point strictness over --ffast-math. It's a genuine win for safety, but that's overkill in some domains (e.g. graphics, games, audio).

Re: Everyone should know SIMD

#233

Post by rao-v »

t-3 wrote: post #200
brabel wrote: post #174

Earlier quoted context omitted.

I think I am missing the joke here. Who exactly is worse off than 11 years ago?

Eggs 11 years ago cost $0.89 (hell, TWO years ago!). Today they cost $5+.

Food inflation has been moderately high but egg prices were a mix of weird short term events and umm price fixing (look up the recent case).

They are back to pre 2022 levels now! https://fred.stlouisfed.org/series/APU0000708111

Re: Everyone should know SIMD

#234

Post by eoerl »

randusername wrote: post #213
kiaansaraiya wrote: post #62

I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.

> Learning to check the compliers optimization reports is arguably more valuable Where do I start? I want to trust the compiler, but I don't always have time to feed every little piece into compiler explorer and interpret it. Is there a higher-level workflow?

this is amazing https://godbolt.org/, you can just paste a function or a bunch of them and instantly see what is generated. It doesn´t take being an expert to start to recognize the (auto) vectorized bits

Re: Everyone should know SIMD

#236

Post by Archit3ch »

randusername wrote: post #213
kiaansaraiya wrote: post #62

I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.

> Learning to check the compliers optimization reports is arguably more valuable Where do I start? I want to trust the compiler, but I don't always have time to feed every little piece into compiler explorer and interpret it. Is there a higher-level workflow?

You can write Julia code as normal and see each compiled function at the REPL. Very similar to a local Compiler Explorer.

Although as mentioned plenty of times, the largest wins tend to come from laying out your data correctly, or switching to a different algorithm. And heuristics. ;)

Re: Everyone should know SIMD

#237

Post by lz400 »

spider-mario wrote: post #219
lz400 wrote: post #84

Isn't the better abstraction here to use a higher level library in the style of pandas/polars that will operate as vectors, compose and feel readable and inuitive, while (almost?) maxing out SIMD?

That can easily cause you to traverse your data several times when once would be enough. Let’s say you wanted to compute “mean of array divided by max in absolute value”. NumPy-like: mean = np.mean(array) maximum = np.max(np.abs(array) return mean / maximum As far as I’m aware, that will be evaluated as three traversals. Highway: HWY_FULL(float) d; using V = decltype(hn::Zero(d)); V sum = hn::Zero(d); V max = hn::Zer…

Thanks, that's insightful. I haven't thought about it that way but in retrospect it's clear. In practice I think we've all seen that the numpy style construction is quick to write and performs (and reads) much better than "dumb loops", but if you really want to optimize your code, the highway version will give you more control (and will read similar to the dumb loop with some decorations). I guess just more tools in the toolbox!

Re: Everyone should know SIMD

#238

Post by abbeyj »

mitchellh wrote: post #11
ashton314 wrote: post #4

Earlier quoted context omitted.

In the article, Mitchel mentions how this doesn’t always work. In fact, as someone who’s worked in compiler development, I can say it’s a small miracle when it does work.

Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.

You would have to give the compiler some help to allow it to auto-vectorize this. Warning, untested: https://godbolt.org/z/b358bMWzG. This unrolls the loop by 16 times. The trick is using `&` instead of `and` so that there's no short-circuiting. All 16 elements are read on each iteration of the loop. This gives the compiler the freedom to replace these reads with a single 16 byte load.

> compilers will never auto-vectorize loops with an early loop break afaik.

I think this is changing. https://godbolt.org/z/ea1E7dx9v. GCC 14 won't try to vectorize this because of the break statement. But GCC 15 does vectorize it. This got a callout in the "General Improvements" section of the release notes: https://gcc.gnu.org/gcc-15/changes.html

I don't think there's any equivalent in clang/LLVM.

Re: Everyone should know SIMD

#239

Post by yunnpp »

spider-mario wrote: post #225
yunnpp wrote: post #54

Earlier quoted context omitted.

You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder You start the post with: > There is an opportunity to use SIMD. SIMD turns those into this: > > for (8 byte chunk in bytes) { /* ... */ } If you actually wrote that loop, there is a good chance the compiler (gcc specifically) wi…

> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand. Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins > I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a p…

How is that different from any other compiler optimization?

And what does the last statement in the quote mean anyway? When is performance "predictable"; do you freeze the entire toolchain?

And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?

If you're writing C++, you're already rolling on decades of compiler optimization. You return by value because it makes code more readable and safer and rely on RVO. You write functions to abstract and rely on the compiler inlining. When it doesn't work for your specific target/toolchain, you may decide to handroll stuff. The proof that you're banking on the compiler is that if you run a debug build of any non-trivial program, it runs like absolute dogshit.

Re: Everyone should know SIMD

#240

Post by arendtio »

Everyone should know SIMD, okay, but please don't introduce that kind of code into my codebase for a linear 5x improvement.

If you work on a performance-critical application, that's fine, but most of the time we use some interpreted / VM / garbage-collected slow environment that performs so much slower than just using hardware directly, and we are fine with it. Under those circumstances, it is much better to have code that every developer can understand than to introduce a 5x improvement, with code that only the person who wrote it understands.

I mean, e.g. for Go, this simplicity was one of the explicit design goals. For C / Rust / Zig, it might be a different story, but I hope you choose those languages only if they fit your use-case.

Post reply on HN