Or... You know... Tell your favorite agent to "SIMD this" (:
Everyone should know SIMD
111–120 of 263 posts
Re: Everyone should know SIMD
#112It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…
It's computer vision focused and might have been suggested previously, but I think Halide is a pretty good/mature demonstration of one way to approach this - writing the algorithm and the execution descriptions as separate passes with access to auto-optimisers and GPU runtimes.
Re: Everyone should know SIMD
#113Good article!
I just wouldn't start off with bold sentences as
> SIMD can be simple to understand
and
> writing SIMD is just about as easy as a for loop
and then the first example requires 12 lines to replace one line of scalar code.
Be honest and say SIMD is hard but the results are worth it!
(Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is good: scalar tails are mentioned and described. Step 1 is bad: nobody is supposed to know what broadcast mean.)
Re: Everyone should know SIMD
#114Remember that bug with Intel Skylakes [0]?
When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure.
Re: Everyone should know SIMD
#115I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.
That’s exactly what happened: https://xcancel.com/mitchellh/status/2079672171321081908#m
You know, it's always funny to read takes like "A broken compiler forcing you to write explicit SIMD instead of trusting auto-vectorization and coming out 20-30% faster is the best argument I've seen for reading your own generated assembly occasionally instead of assuming the compiler has you covered" because you can quite easily imagine an alternative one like "A broken compiler revealing that the auto-vectorization actually already accounts for 50% of total speed up of O3, and manual reimplementation and code restructuring provided only additional 20% in some scenarios is the best argument I've seen for almost never bothering with hand-crafting assembly anymore".
Re: Everyone should know SIMD
#116Earlier quoted context omitted.
Yeah, data layout/cache aware layouts are really key if you really want to unlock making something that ends up in a hot loop fast with SIMD. Also, avoiding allocations or vtable lookups or a lot of indirection in the part of the code that's actually "hot" is really important. Vectors (in C++) at least aren't necessarily the best fit either, if you end up doing anything that can call an allocation unexpectedly.
> Vectors (in C++) at least aren't necessarily the best fit either I'm not sure if you use a different allocation strategy or if you're advocating allocating as much as possible up-front, but I'm curious if you have any thoughts on this: I always end up using (Rust) vectors despite looking at a bunch of slab/arena allocation libraries. Preferably I'd know how much memory I need up front, but barring that I see three…
The main reason for virtual addresses/TLB is to prevent processes from accessing each other's memory (isolation). For optimization I'd say the page size (64 kB) is a secondary concern. It's handled in hardware (TLB) with the OS only occasionally filling up the mapping. You want to avoid that happening, but you probably should worry more using whole cache lines (64 bytes) instead, and beyond that just keep memory access local (multiple cache hierarchies) and predictable (pre-fetcher).
Re: Everyone should know SIMD
#117Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqY
It's a great talk, I just wish there was a good focused textual version of it, as it is a very long video to recommend to others. Very worth it, but a big investment.
It's a great example of what I think of as vertical integration for performance. As you go through the talk you can understand why all these abstractions exist and why they have to be so generic. But when you have a specific use case, you can vertically integrate from the problem definition all the way down to SIMD and reap big rewards.
Re: Everyone should know SIMD
#118Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…
This is probably one of the biggest sins in technological teaching. Sure it is crucially important to take away the fear of a topic. But you don't do so by saying it is simple, you do so by showing it is simple.
And it turns out sometimes you cannot show it is simple, because it is in fact very complex. But every complex topic is made up of smaller, simpler ones. Good teachers then manage to find a good order of those smaller parts that makes the steep hill climbable. Then you only need to convince people it is actually worth climbing.
Re: Everyone should know SIMD
#119Remember that bug with Intel Skylakes [0]? When an application used AVX, it slowed down everything else on that node. It was by far not easy to debug why some applications randomly suffered perf hits on a new hardware being rolled out in Azure. [0] https://arxiv.org/abs/1901.04982?utm_source=chatgpt.com
It was an early AVX-512 unit that resulted in down lock when you used more than a few instructions in short time, as well as resulted in a Linus rant.
Later CPUs (updated skylake xeons and later, AMD Zen 4 and newer) don't have the issue
Re: Everyone should know SIMD
#120> Every developer should know at least that much SIMD.
> This [...] applies to any programming language. Support for SIMD instructions varies by programming language
This is a very pedantic nitpick because this article is good (& getting SIMD support across more languages would also be good), but the "every programmer should know" line feels a bit odd when neither of the 2 most popular languages natively support SIMD.