Fundamental flaws of SIMD ISAs (2021) • Hacker News Board

Re: Fundamental flaws of SIMD ISAs (2021)

#81

Post by adgjlsfhk1 » Fri, Apr 25, 2025, 6:30 AM UTC

ack_complete wrote: post #73

adgjlsfhk1 wrote: post #67

Earlier quoted context omitted.

It will be interesting seeing what happens now that AMD is shipping good AVX-512. It really just makes Intel seem incompetent (especially since they're theoretically bringing AVX-512 back in next year anyway)

No proof, but I suspect that AMD's AVX-512 support played a part in Intel dumping AVX10/256 and changing plans back to shipping a full 512-bit consumer implementation again (we'll see when they actually ship it). The downside is that AMD also increased the latency of all formerly cheap integer vector ops. This removes one of the main advantages against NEON, which historically has had richer operations but worse late…

> For instance, Skymont finally has no penalty for denormals, a long standing Intel weakness.

yeah, that's crazy to me. Intel has been so completely discunctional for the last 15 years. I feel like you couldn't have a clearer sign of "we have 2 completely separate teams that are competing with each other and aren't allowed to/don't want to talk to each other". it's just such a clear sign that the chicken is running around headless

Re: Fundamental flaws of SIMD ISAs (2021)

#82

Post by adgjlsfhk1 » Fri, Apr 25, 2025, 6:34 AM UTC

sweetjuly wrote: post #77

convolvatron wrote: post #35

Earlier quoted context omitted.

kinda? this is sort of a direct result of the 'vectors are just sliced registers' model. if i do a pairwise operation and divide my domain by 2 at each step, is the resulting vector sparse or dense? if its dense then I only really top out when i'm in the last log2slice steps.

Yes, but this is not cheap for hardware. CPU designers love SIMD because it lets them just slap down ALUs in parallel and get 32x performance boosts. Reductions, however, are not entirely parallel and instead have a relatively huge gate depth. For example, suppose an add operation has a latency of one unit in some silicon process. To add reduce a 32 element vector, you'll have a five deep tree, which means your opera…

for some, you can cheat a little (e.g. partial sum reduction), but then you don't get to share IP with the rest of your ALUs. I do really want to see what an optimal 32 wide reduction and circuit looks like. For integers, you pretty clearly can do much better than pairwise reduction. float sounds tricky

Re: Fundamental flaws of SIMD ISAs (2021)

#83

Post by codedokode » Fri, Apr 25, 2025, 7:17 AM UTC

I think that packed SIMD is better in almost every aspect and Vector SIMD is worse.

With vector SIMD you don't know the register size beforehand and therefore have to maintain and increment counters, adding extra unnecessary instructions, reducing total performance. With packed SIMD you can issue several loads immediately without dependencies, and if you look at code examples, you can see that the x86 code is more dense and uses a sequence of unrolled SIMD instructions without any extra instructions which is more efficient. While RISC-V has 4 SIMD instructions and 4 instructions dealing with counters per loop iteration, i.e. it wastes 50% of command issue bandwidth and you cannot load next block until you increment the counter.

The article mentions that you have to recompile packed SIMD code when a new architecture comes out. Is that really a problem? Open source software is recompiled every week anyway. You should just describe your operations in a high level language that gets compiled to assembly for all supported architectures.

So as a conclusion, it seems that Vector SIMD is optimized for manually-written assembly and closed-source software while Packed SIMD is made for open-source software and compilers and is more efficient. Why RISC-V community prefers Vector architecture, I don't understand.

Re: Fundamental flaws of SIMD ISAs (2021)

#84

Post by codedokode » Fri, Apr 25, 2025, 7:19 AM UTC

__abadams__ wrote: post #55

kevingadd wrote: post #34

Earlier quoted context omitted.

Part of the motive behind variable width SIMD in WASM is that there's intentionally-ish no mechanism to do feature detection at runtime in WASM. The whole module has to be valid on your target, you can't include a handful of invalid functions and conditionally execute them if the target supports 256-wide or 512-wide SIMD. If you want to adapt you have to ship entire modules for each set of supported feature flags and…

It would be very easy to support 512-bit vectors everywhere, and just emulate them on most systems with a small number of smaller vectors. It's easy for a compiler to generate good code for this. Clang does it well if you use its built-in vector types (which can be any length). Variable-length vectors, on the other hand, are a very challenging problem for compiler devs. You tend to get worse code out than if you just…

Variable length vectors seem to be made for closed-source manually-written assembly (nobody wants to unroll the loop manually and nobody will rewrite it for new register width).

Re: Fundamental flaws of SIMD ISAs (2021)

#85

Post by codedokode » Fri, Apr 25, 2025, 7:21 AM UTC

derf_ wrote: post #49

TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

> I code with SIMD as the target, and have special containers that pad memory to SIMD width... I think this may be domain-specific. I help maintain several open-source audio libraries, and wind up being the one to review the patches when people contribute SIMD for some specific ISA, and I think without exception they always get the tail handling wrong. Due to other interactions it cannot always be avoided by padding.…

I think that SIMD code should not be written by hand but rather in a high-level language and so dealing with tail becomes a compiler's and not a programmer's problem. Or people still prefer to write assembly be hand? It seems to be so judging by the code you link.

What I wanted is to write code in a more high-level language like this. For example, to compute a scalar product of a and b you write:

    1..n | a[$1] * b[$1] | sum

Or maybe this:

    x = sum for i in 1 .. n: a[i] * b[i]

And the code gets automatically compiled into SIMD instructions for every existing architecture (and for large arrays, into a multi-thread computation).

Re: Fundamental flaws of SIMD ISAs (2021)

#86

Post by [deleted] » Fri, Apr 25, 2025, 7:28 AM UTC

derf_ wrote: post #49

TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

> I code with SIMD as the target, and have special containers that pad memory to SIMD width... I think this may be domain-specific. I help maintain several open-source audio libraries, and wind up being the one to review the patches when people contribute SIMD for some specific ISA, and I think without exception they always get the tail handling wrong. Due to other interactions it cannot always be avoided by padding.…

[deleted]

Re: Fundamental flaws of SIMD ISAs (2021)

#87

Post by synthos » Fri, Apr 25, 2025, 7:35 AM UTC

jandrewrogers wrote: post #59

camel-cdr wrote: post #29

Earlier quoted context omitted.

> The argument for vector width agnostic code is seems predicated on the proverbial “sufficiently advanced compiler”. A SIMD ISA having a fixed size or not is orthogonal to autovectorization. E.g. I've seen a bunch of cases where things get autovectorized for RVV but not for AVX512. The reason isn't fixed vs variable, but rather the supported instructions themselves. There are two things I'd like from a "sufficiently…

The interplay of SIMD width and microarchitecture is more important for performance engineering than you seem to be assuming. Those codegen decisions are made at layer above anything being talked about here and they operate on explicit awareness of things like register size. It isn’t “same instruction but wider or narrower” or anything that can be trivially autovectorized, it is “different algorithm design”. Compiler…

> Compilers are not yet rewriting data structures and algorithms based on microarchitecture.

Afaik mojo allows for this with the autotuning capability and metaprogramming

Re: Fundamental flaws of SIMD ISAs (2021)

#88

Post by codedokode » Fri, Apr 25, 2025, 7:44 AM UTC

camel-cdr wrote: post #19

aengelke wrote: post #13

Earlier quoted context omitted.

I agree; and the article seems to have also quite a few technical flaws: - Register width: we somewhat maxed out at 512 bits, with Intel going back to 256 bits for non-server CPUs. I don't see larger widths on the horizon (even if SVE theoretically supports up to 2048 bits, I don't know any implementation with ~~>256~~ >512 bits). Larger bit widths are not beneficial for most applications and the few applications tha…

> we somewhat maxed out at 512 bits Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. Also there are processors with larger vector length, e.g. 1024-bit: Andes AX45MPV, SiFive X380, 2048-bit: Akeana 1200, 16384-bit: NEC SX-Aurora, Ara, EPI > no way around this You rarely need to rewrite SIMD code to take advantage of new extensions, unless som…

> Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA.

This is a wrong approach. You should be writing you code in a high-level language like this:

    x = sum i for 1..n: a[i] * b[i]

And let the compiler write the assembly for every existing architecture (including multi-threaded version of a loop).

I don't understand what is the advantage of writing the SIMD code manually. At least have a LLM write it if you don't like my imaginary high-level vector language.

Re: Fundamental flaws of SIMD ISAs (2021)

#89

Post by saagarjha » Fri, Apr 25, 2025, 7:52 AM UTC

gatane wrote: post #61

PaulHoule wrote: post #52

Earlier quoted context omitted.

At the very least you can decode UTF-8 really quickly with AVX-512 https://lemire.me/blog/2023/08/12/transcoding-utf-8-strings-... and web browsers at the very least spent a lot of cycles on decoding HTML and Javascript which is UTF-8 encoded. It turns out AVX-512 is good at a lot of things you wouldn't think SIMD would be good at. Intel's got the problem that people don't want to buy new computers because they don't…

In the end, it doesnt even matter, javascript frameworks are already big enough to slow down your pc. Unless if said optimization on parsing runs at the very core of JS.

It'll speed up first load times.

Re: Fundamental flaws of SIMD ISAs (2021)

#90

Post by codedokode » Fri, Apr 25, 2025, 7:53 AM UTC

xphos wrote: post #74

Personally, I think load and increment address register in a single instruction is extremely valuable here. It's not quite the risc model but I think that it is actually pretty significant in avoiding a von nurmon bottleneck with simd (the irony in this statement) I found that a lot of the custom simd cores I've written for simply cannot issue instructions fast enough risvc. Or when they it's in quick bursts and than…

> load and increment address register in a single instruction is extremely valuable here

I think this is not important anymore because modern architectures allow to add offset to register value so you can write something like, using weird RISC-V syntax for addition:

    ld r2, 0(r1)
    ld r3, 4(r1)
    ld r4, 8(r1)

These operations can be executed in parallel, while with auto-incrementing you cannot do that.