Live data from Hacker News

Everyone should know SIMD

mitchellh.com

161–170 of 263 posts

Re: Everyone should know SIMD

#161
post #119
post #114

Remember that bug with Intel Skylakes [0]? When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure. [0] https://arxiv.org/abs/1901.04982?utm_source=chatgpt.com

It was an early AVX-512 unit that resulted in down lock when you used more than a few instructions in short time, as well as resulted in a Linus rant. Later CPUs (updated skylake xeons and later, AMD Zen 4 and newer) don't have the issue

Historically, it was a bit more complex than that, and applies to more than just AVX-512: https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e0....

Re: Everyone should know SIMD

#162

Does anyone know of a good hands-on introductory and practical tutorial on SIMD? I know that SIMD doesn't imply particular instruction set but a pattern of concurrent data transformation. I guess what I'm looking for is something that addresses the common idioms in SIMD. For example, want to do concurrent data look up? This is how you do it in SIMD; this is how you search; this is how to prepare your data in a manner…

> I know that SIMD doesn't imply particular instruction set but a pattern of concurrent data transformation.

Eh. It's both. You don't have one without the other. SIMD is pretty much entirely extensions to base processor ISA; there might be a few architectures that have SIMD as a basic part of them (GPUs are one of them, taken to an extreme order), but for the most part you have to know which instruction set you're working with.

SIMD is basically "pack multiple data points into a single CPU register, then to another, and perform some operation on them as if you ran that op on all of the pairwise data points individually". Some ops are binary (arithmetic, bitwise, etc) and some are unary.

Some have "gates" whereby you can do a comparison, the boolean output of which is stored in a bit packed integer. Then you can run conditional instructions after that that only perform the instruction if the bit in that variable is set, creating "constant time" SIMD instructions with what amounts to branching. Really depends on the instruction set's capabilities.

AVX512 is far and away one of the most extensive extensions, with a ton of super niche instructions meant for enterprise number crunching. It has weird stuff, like swapping bytes, collating them, doing all sorts of weird manipulations. But the number of x86 CPUs that support that instruction set are small.

You can't emit code that has e.g. AVX512 and just run it on a CPU that doesn't have that extension. You get a CPU exception and it crashes the process (or, if this is in kernel/driver land, your machine).

So they're kind of tied together.

Anyway if you want to see a list of them, Intel's SIMD intrinsics site has always been really nice to browse IMO.

https://www.intel.com/content/www/us/en/docs/intrinsics-guid...

Re: Everyone should know SIMD

#163

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

Agreed, I was interested and I'm prob the target audience but things escalated too fast too quickly, very similar to the infamous "how to draw an owl" meme.

Re: Everyone should know SIMD

#164
post #147

The example code in this article is using Zig's portable SIMD features. Similar features are available for C/C++ (GCC/Clang extension) [0], (nightly) Rust [1] and C++26 [2]. All of these provide a similar set of features and you can use normal arithmetic operations (+, -, *, etc) for SIMD vectors. Together with templates/generics you can also write code that can deal with any vector width. These get compiled to LLVM…

FYI you linked to a really old version of the GCC documentation. Google apparently loves those old docs, so they often show up near the top of search results despite being ancient. For posterity, here's the latest version: https://gcc.gnu.org/onlinedocs/gcc-16.1.0/gcc/Vector-Extensi....

Re: Everyone should know SIMD

#166

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

> Okay, now I understand that those 12 lines are going to look really alien to someone not familiar with the concepts. So now let's back up and explain it step by step, mapping it directly to the shape previously mentioned.

They covered that, in a very honest and blunt manner.

Re: Everyone should know SIMD

#168

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

Around 1990 I had the fortune to learn Parallel-C - a language that was designed during the Transputer hype and was essentially C extended by a few features to support easy parallel programming.

My favourite feature was

  par(; ; )
essentially a for loop that will be auto-parallelized by the compiler - some boundary conditions apply.

Re: Everyone should know SIMD

#169
post #114

Remember that bug with Intel Skylakes [0]? When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure. [0] https://arxiv.org/abs/1901.04982?utm_source=chatgpt.com

The Intel server CPUs Skylake Server, Cascade Lake and Cooper Lake, which had bad frequency/voltage management are now ancient history and very few of them have been used as workstation CPUs by individual users.

AMD Zen 4 and Zen 5, and also those Intel CPUs with AVX-512 support starting with Ice Lake, behave much better and there is no reason to avoid AVX-512, which has much better energy efficiency than the alternatives.

Re: Everyone should know SIMD

#170
And here I am pooping out simple typescript... These articles always make me feel like I'm wasting my talents working on products that don't really have the need to leverage any understanding of what's happening under the hood at the CPU instructions level.
Post reply on HN