Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fundamental flaws of SIMD ISAs (2021)

bitsnbites.eu

Post reply on HN
Print view View original

101–110 of 146 posts

Re: Fundamental flaws of SIMD ISAs (2021)

#101

Post by IshKebab »

tonetegeatinst wrote: post #26
TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

AFAIK about every modern CPU uses out of order von Neumann architecture. The only people who don't are the handful of researchers and people who work with the government research into non van Neumann designed systems.

Microcontrollers are often in-order.

Re: Fundamental flaws of SIMD ISAs (2021)

#102

Post by Someone »

__abadams__ wrote: post #55
kevingadd wrote: post #34

Earlier quoted context omitted.

Part of the motive behind variable width SIMD in WASM is that there's intentionally-ish no mechanism to do feature detection at runtime in WASM. The whole module has to be valid on your target, you can't include a handful of invalid functions and conditionally execute them if the target supports 256-wide or 512-wide SIMD. If you want to adapt you have to ship entire modules for each set of supported feature flags and…

It would be very easy to support 512-bit vectors everywhere, and just emulate them on most systems with a small number of smaller vectors. It's easy for a compiler to generate good code for this. Clang does it well if you use its built-in vector types (which can be any length). Variable-length vectors, on the other hand, are a very challenging problem for compiler devs. You tend to get worse code out than if you just…

> It would be very easy to support 512-bit vectors everywhere, and just emulate them on most systems with a small number of smaller vectors. It's easy for a compiler to generate good code for this

Wouldn’t that be suboptimal if/when CPUs that support 1024-bit vectors come along?

> Variable-length vectors, on the other hand, are a very challenging problem for compiler devs. You tend to get worse code out than if you just statically picked a size, even if it's not the native size.

Why would it be challenging? You could statically pick a size on a system with variable-length vectors, too. How would that be worse code?

Re: Fundamental flaws of SIMD ISAs (2021)

#103

Post by stephencanon »

The basic problem with almost every "SIMD is flawed, we should have vector ISAs" article or post (including the granddaddy, "SIMD Instructions Considered Harmful"), is that they invariably use SAXPY or something else trivial where everything stays neatly in lane as their demonstration case. Of course vector ISAs look good when you show them off using a pure vector task. This is fundamentally unserious.

There is an enormous quantity of SIMD code in the world that isn't SAXPY, and doesn't stay neatly in lane. Instead it's things like "base64 encode this data" or "unpack and deinterleave this 4:2:2 pixel data, apply a colorspace conversion as a 3x3 sparse matrix and gamma adjustment in 16Q12 fixed-point format, resize and rotate by 15˚ with three shear operations represented as a linear convolution with a sinc kernel per row," or "extract these fields from this JSON data". All of which _can totally be done_ with a well-designed vector ISA, but the comparison doesn't paint nearly as rosy of a picture. The reality is that you really want a mixture of ideas that come from fixed-width SIMD and ideas that come from the vector world (which is roughly what people actually shipping hardware have been steadily building over the last two decades, implementing more support for unaligned access, predication, etc, while the vector ISA crowd writes purist think pieces)

Re: Fundamental flaws of SIMD ISAs (2021)

#104

Post by IshKebab »

codedokode wrote: post #83

I think that packed SIMD is better in almost every aspect and Vector SIMD is worse. With vector SIMD you don't know the register size beforehand and therefore have to maintain and increment counters, adding extra unnecessary instructions, reducing total performance. With packed SIMD you can issue several loads immediately without dependencies, and if you look at code examples, you can see that the x86 code is more de…

Those 4 counter instructions have no dependencies though so they'll likely all be issued and executed in parallel in 1 cycle, surely? Probably the branch as well in fact.

Re: Fundamental flaws of SIMD ISAs (2021)

#105

Post by pornel »

almostgotcaught wrote: post #80
pornel wrote: post #6

There are alternative universes where these wouldn't be a problem. For example, if we didn't settle on executing compiled machine code exactly as-is, and had a instruction-updating pass (less involved than a full VM byte code compilation), then we could adjust SIMD width for existing binaries instead of waiting decades for a new baseline or multiversioning faff. Another interesting alternative is SIMT. Instead of hav…

> There are alternative universes where these wouldn't be a problem Do people that say these things have literally any experience of merit? > For example, if we didn't settle on executing compiled machine code exactly as-is, and had a instruction-updating pass You do understand that at the end of the day, hardware is hard (fixed) and software is soft (malleable) right? There will be always be friction at some boundar…

The point of updating the instructions isn't to have optimal behavior in all cases, or to reconfigure programs for wildly different hardware, but to be able to easily target contemporary hardware, without having to wait for the oldest hardware to die out first to be able to target a less outdated baseline without conditional dispatch.

Users are much more forgiving about software that runs a bit slower than software that doesn't run at all. ~95% of x86_64 CPUs have AVX2 support, but compiling binaries to unconditionally rely on it makes the remaining users complain. If it was merely slower on potato hardware, it'd be an easier tradeoff to make.

This is the norm on GPUs thanks to shader recompilation (they're far from optimal for all hardware, but at least get to use the instruction set of the HW they're running on, instead of being limited to the lowest common denominator). On CPUs it's happening in limited cases: Zen 3 added AVX-512 by executing two 256-bit operations serially, and plenty of less critical instructions are emulated in microcode, but it's done by the hardware, because our software isn't set up for that.

Compilers already need to make assumptions about pipeline widths and instruction latencies, so the code is tuned for specific CPU vendors/generations anyway, and that doesn't get updated. Less explicitly, optimized code also makes assumptions about cache sizes and compute vs memory trade-offs. Code may need L1 cache of certain size to work best, but it still runs on CPUs with a too-small L1 cache, just slower. Imagine how annoying it would be if your code couldn't take advantage of a larger L1 cache without crashing on older CPUs. That's where CPUs are with SIMD.

Re: Fundamental flaws of SIMD ISAs (2021)

#106

Post by Const-me »

Remnant44 wrote: post #38
sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

These days, I strongly believe that loop unrolling is a pessimization, especially with SIMD code. Scalar code should be unrolled by the compiler to the SIMD word width to expose potential parallelism. But other than that, correctly predicted branches are free, and so is loop instruction overhead on modern wide-dispatch processors. For example, even running a maximally efficient AVX512 kernel on a zen5 machine that di…

> schedule the operations identically whether you did one copy per loop or four

They don’t always do that well when you need a reduction in that loop, e.g. you are searching for something in memory, or computing dot product of long vectors.

Reductions in the loop form a continuous data dependency chain between loop iteration, which prevents processors from being able to submit instructions for multiple iterations of the loop. Fixable with careful manual unrolling.

Re: Fundamental flaws of SIMD ISAs (2021)

#107

Post by Const-me »

TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

> I prefer fixed width

Another reason to prefer fixed width, compilers may pass vectors to functions in SIMD registers. When register size is unknown at compile time, they have to pass data in memory. For complicated SIMD algorithms the performance overhead gonna be huge.

Re: Fundamental flaws of SIMD ISAs (2021)

#108

Post by codedokode »

IshKebab wrote: post #104
codedokode wrote: post #83

I think that packed SIMD is better in almost every aspect and Vector SIMD is worse. With vector SIMD you don't know the register size beforehand and therefore have to maintain and increment counters, adding extra unnecessary instructions, reducing total performance. With packed SIMD you can issue several loads immediately without dependencies, and if you look at code examples, you can see that the x86 code is more de…

Those 4 counter instructions have no dependencies though so they'll likely all be issued and executed in parallel in 1 cycle, surely? Probably the branch as well in fact.

The load instruction has a dependency on counter increment. While with packed SIMD one can issue several loads without waiting. Also, extra counter instructions still waste resources of a CPU (unless there is some dedicated hardware for this purpose).

Re: Fundamental flaws of SIMD ISAs (2021)

#109

Post by kjs3 »

immibis wrote: post #94
PaulHoule wrote: post #52

Earlier quoted context omitted.

At the very least you can decode UTF-8 really quickly with AVX-512 https://lemire.me/blog/2023/08/12/transcoding-utf-8-strings-... and web browsers at the very least spent a lot of cycles on decoding HTML and Javascript which is UTF-8 encoded. It turns out AVX-512 is good at a lot of things you wouldn't think SIMD would be good at. Intel's got the problem that people don't want to buy new computers because they don't…

If you pay attention this isn't a UTF-8 decoder. It might be some other encoding, or a complete misunderstanding of how UTF-8 works, or an AI hallucination. It also doesn't talk about how to handle the variable number of output bytes or the possibility of a continuation sequence split between input chunks.

I paid attention and I don't see where Daniel claimed that this a complete UTF-8 decoder. He's illustrating a programming technique using a simplified use case, not solving the worlds problems. And I don't think Daniel Lemire lacks an understanding of the concept or needs an AI to code it.

Re: Fundamental flaws of SIMD ISAs (2021)

#110

Post by otherjason »

codedokode wrote: post #88
camel-cdr wrote: post #19

Earlier quoted context omitted.

> we somewhat maxed out at 512 bits Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. Also there are processors with larger vector length, e.g. 1024-bit: Andes AX45MPV, SiFive X380, 2048-bit: Akeana 1200, 16384-bit: NEC SX-Aurora, Ara, EPI > no way around this You rarely need to rewrite SIMD code to take advantage of new extensions, unless som…

> Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. This is a wrong approach. You should be writing you code in a high-level language like this: x = sum i for 1..n: a[i] * b[i] And let the compiler write the assembly for every existing architecture (including multi-threaded version of a loop). I don't understand what is the advantage of writin…

This is the common argument from proponents of compiler autovectorization. An example like what you have is very simple, so modern compilers would turn it into SIMD code without a problem.

In practice, though, the cases that compilers can successfully autovectorize are very limited relative to the total problem space that SIMD is solving. Plus, if I rely on that, it leaves me vulnerable to regressions in the compiler vectorizer.

Ultimately for me, I would rather write the implementation myself and know what is being generated versus trying to write high-level code in just the right way to make the compiler generate what I want.

Post reply on HN