Remember that bug with Intel Skylakes [0]? When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure. [0] https://arxiv.org/abs/1901.04982?utm_source=chatgpt.com
It was an early AVX-512 unit that resulted in down lock when you used more than a few instructions in short time, as well as resulted in a Linus rant. Later CPUs (updated skylake xeons and later, AMD Zen 4 and newer) don't have the issue
Everyone should know SIMD
161–170 of 263 posts
Re: Everyone should know SIMD
#162Does anyone know of a good hands-on introductory and practical tutorial on SIMD? I know that SIMD doesn't imply particular instruction set but a pattern of concurrent data transformation. I guess what I'm looking for is something that addresses the common idioms in SIMD. For example, want to do concurrent data look up? This is how you do it in SIMD; this is how you search; this is how to prepare your data in a manner…
Eh. It's both. You don't have one without the other. SIMD is pretty much entirely extensions to base processor ISA; there might be a few architectures that have SIMD as a basic part of them (GPUs are one of them, taken to an extreme order), but for the most part you have to know which instruction set you're working with.
SIMD is basically "pack multiple data points into a single CPU register, then to another, and perform some operation on them as if you ran that op on all of the pairwise data points individually". Some ops are binary (arithmetic, bitwise, etc) and some are unary.
Some have "gates" whereby you can do a comparison, the boolean output of which is stored in a bit packed integer. Then you can run conditional instructions after that that only perform the instruction if the bit in that variable is set, creating "constant time" SIMD instructions with what amounts to branching. Really depends on the instruction set's capabilities.
AVX512 is far and away one of the most extensive extensions, with a ton of super niche instructions meant for enterprise number crunching. It has weird stuff, like swapping bytes, collating them, doing all sorts of weird manipulations. But the number of x86 CPUs that support that instruction set are small.
You can't emit code that has e.g. AVX512 and just run it on a CPU that doesn't have that extension. You get a CPU exception and it crashes the process (or, if this is in kernel/driver land, your machine).
So they're kind of tied together.
Anyway if you want to see a list of them, Intel's SIMD intrinsics site has always been really nice to browse IMO.
https://www.intel.com/content/www/us/en/docs/intrinsics-guid...
Re: Everyone should know SIMD
#163Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…
Re: Everyone should know SIMD
#164The example code in this article is using Zig's portable SIMD features. Similar features are available for C/C++ (GCC/Clang extension) [0], (nightly) Rust [1] and C++26 [2]. All of these provide a similar set of features and you can use normal arithmetic operations (+, -, *, etc) for SIMD vectors. Together with templates/generics you can also write code that can deal with any vector width. These get compiled to LLVM…
Re: Everyone should know SIMD
#165Re: Everyone should know SIMD
#166Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…
They covered that, in a very honest and blunt manner.
Re: Everyone should know SIMD
#167Re: Everyone should know SIMD
#168Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…
My favourite feature was
par(; ; )
essentially a for loop that will be auto-parallelized by the compiler - some boundary conditions apply.Re: Everyone should know SIMD
#169Remember that bug with Intel Skylakes [0]? When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure. [0] https://arxiv.org/abs/1901.04982?utm_source=chatgpt.com
AMD Zen 4 and Zen 5, and also those Intel CPUs with AVX-512 support starting with Ice Lake, behave much better and there is no reason to avoid AVX-512, which has much better energy efficiency than the alternatives.