I was hand-rolling NEON SIMD 15 years ago, and in many cases the compiler (clang/llvm) simply out optimized me. I kept the attempts that were better than what the compiler could already do. That was ARM NEON, quite new at the time, not SSE, and again that was 15 years ago that the compiler could already beat me much of the time. I hate the naive cult coder adage concerning premature optimization, but this might be a situation where you peruse your compiler output before you start writing code in a manner the compiler can for you. In Clang, auto-vectorization is enabled by default at optimization levels -O2 and -O3.
Everyone should know SIMD
31–40 of 263 posts
Re: Everyone should know SIMD
#32Earlier quoted context omitted.
What are AoS and SoA?
Array of Structs and Struct of Arrays https://en.wikipedia.org/wiki/AoS_and_SoA
A good introduction to SoA (for anyone curious) are the two most famous Data-Oriented Design talks by Mike Acton (game engine dev) [1] and Andrew Kelley (Zig lead dev) [2] respectively.
I read a book book about DoD [3] really which confused me at first with all its talk about database table design (in a book about a high-performance C++ game engine?), but when it finally clicked it was amazing. The point is that you want to think hard about your access patterns and what could constitute good "primary keys", then model it accordingly. SoA ends up being useful a lot of the time, because having your data in homogeneous arrays/vectors is great for cache locality and branch elimination. Even without SIMD you can get huge speedups from that, but that's also where your compiler (or you as a programmer) can get incredible SIMD gains.
SoA is not a silver bullet as it may not align well with your access patterns, but it can great to add to your toolkit.
---
Mike Acton: Data-Oriented Design and C++: https://www.youtube.com/watch?v=rX0ItVEVjHc
Andrew Kelley: A Practical Guide to Applying Data Oriented Design: https://www.youtube.com/watch?v=IroPQ150F6c
Richard Fabian: Data-Oriented Design: https://www.dataorienteddesign.com/dodbook/
Re: Everyone should know SIMD
#33> Every developer should… most importantly, not be scared of SIMD
Seems like he should be recommending fearless_simd [1], the Rust crate by Raph Levian and the folks at Linebender :)
More seriously, if you’re looking to add SIMD to your Rust code, that’s the package to start with.
Re: Everyone should know SIMD
#3499% of developers should just ignore SIMD. Most projects have a lot of low hanging fruit to increase performance, and still nobody finds the time to solve them.
Re: Everyone should know SIMD
#35[flagged]
Re: Everyone should know SIMD
#36[deleted]
Re: Everyone should know SIMD
#37I just do gcc -O3 and get SIMD without having to learn it
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers are really good at vectorizing on whatever hardware you're targeting, including SVE.
Re: Everyone should know SIMD
#38I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns. I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was l…
Yeah, data layout/cache aware layouts are really key if you really want to unlock making something that ends up in a hot loop fast with SIMD. Also, avoiding allocations or vtable lookups or a lot of indirection in the part of the code that's actually "hot" is really important. Vectors (in C++) at least aren't necessarily the best fit either, if you end up doing anything that can call an allocation unexpectedly.
> Vectors (in C++) at least aren't necessarily the best fit either
I'm not sure if you use a different allocation strategy or if you're advocating allocating as much as possible up-front, but I'm curious if you have any thoughts on this:
I always end up using (Rust) vectors despite looking at a bunch of slab/arena allocation libraries. Preferably I'd know how much memory I need up front, but barring that I see three options for any allocation that needs to grow:
- Fail;
- Reallocate; or
- Put the overflow in a new allocation, keeping track of where all the "pages" are internally.
In 2, you can't use references/slices (anything with a pointer) because the potential reallocation invalidates those. In 3, it seems ideal because references can stay stable, but you wouldn't be able to have any array-like data cross the "page" barrier as the pointer jump would not be stable. (Although, am I correct in thinking the OS does something like this, and that's why pointer addresses are virtual?).
So, you can't really internally reference data in this sort of context by reference/pointer, it's preferable to use an integer index. In that case, what's the point of the allocator libraries at all? Your standard Vector would have the same reallocation/access characteristics, and you can set a reasonable initial capacity to try and avoid reallocations.
Re: Everyone should know SIMD
#39"More importantly, when this loop matters enough for me to care about a 5x speedup, I want the vectorization to be explicit and predictable. I don't want an unrelated code change or compiler update to quietly turn it back into a scalar loop." Only tangentially related but this is by far the most painful part about optimizing code for JIT compilers like V8. Even changing a constant from 1 to 1.0 somewhere else can cha…
Proving the point that the compilers aren't deterministic as some folks argue.
This is especially painful with dynamic languages, like in JavaScript's case.
However JIT also have positives hence their widespread use.
Re: Everyone should know SIMD
#40Not really, not as SIMD remains an esoteric art from packing matrix and vector operations in endless opcodes.
I rather let the compiler auto vectorise itself, or with AI help.