Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fundamental flaws of SIMD ISAs (2021)

bitsnbites.eu

Post reply on HN
Print view View original

31–40 of 146 posts

Re: Fundamental flaws of SIMD ISAs (2021)

#31

Post by camel-cdr »

aengelke wrote: post #24
camel-cdr wrote: post #19

Earlier quoted context omitted.

> we somewhat maxed out at 512 bits Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. Also there are processors with larger vector length, e.g. 1024-bit: Andes AX45MPV, SiFive X380, 2048-bit: Akeana 1200, 16384-bit: NEC SX-Aurora, Ara, EPI > no way around this You rarely need to rewrite SIMD code to take advantage of new extensions, unless som…

> Also there are processors with larger vector length How do these fare in terms of absolute performance? The NEC TSUBASA is not a CPU. > Do you have more examples of this? I ported some numeric simulation kernel to the A64Fx some time ago, fixing the vector width gave a 2x improvement. Compilers probably/hopefully have gotten better in the mean time and I haven't redone the experiments since then, but I'd be surpris…

> How do these fare in terms of absolute performance? The NEC TSUBASA is not a CPU.

The NEC is an attached accelerator, but IIRC it can run an OS in host mode. It's hard to tell how the others perform, because most don't have hardware available yet or only they and partner companies have access. It's also hard to compare, because they don't target the desktop market.

> I ported some numeric simulation kernel to the A64Fx some time ago, fixing the vector width gave a 2x improvement.

Oh, wow. Was this autovectorized or handwritten intrinsics/assembly?

Any chance it's of a small enough scope that I could try to recreate it?

> I was specifically referring to dynamic vector sizes.

Ah, sorry, yes you are correct. It still shows that supporting VLA mechanisms in an ISA doesn't mean it's slower for fixed-size usage.

I'm not aware of any proper VLA vs VLS comparisons. I benchmarked a VLA vs VLS mandelbrot implementation once where there was no performance difference, but that's a too simple example.

Re: Fundamental flaws of SIMD ISAs (2021)

#32

Post by vardump »

adgjlsfhk1 wrote: post #27
vardump wrote: post #22

Earlier quoted context omitted.

> Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. 256 and 512 bits are the only reasonable widths. 256 bit AVX2 is what, 13 or 14 years old now.

no. Because Intel is full of absolute idiots, Intel atom didn't support AVX 1 until Gracemont. Tremont is missing AVX1, AVX2, FMA, and basically the rest of X86v3, and shipped in CPUs as recently as 2021 (Jasper Lake).

Oh damn. I've dropped SSE ages ago and no one complained. I guess the customer base didn't use those chips...

Re: Fundamental flaws of SIMD ISAs (2021)

#33

Post by timewizard »

> Another problem is that each new SIMD generation requires new instruction opcodes and encodings.

It requires new opcodes. It does not strictly require new encodings. Several new encodings are legacy compatible and can encode previous generations vector instructions.

> so the architecture must provide enough SIMD registers to avoid register spilling.

Or the architecture allows memory operands. The great joy of basic x86 encoding is that you don't actually need to put things in registers to operate on them.

> Usually you also need extra control logic before the loop. For instance if the array length is less than the SIMD register width, the main SIMD loop should be skipped.

What do you want? No control overhead or the speed enabled by SIMD? This isn't a flaw. This is a necessary price to achieve the efficiency you do in the main loop.

Re: Fundamental flaws of SIMD ISAs (2021)

#34

Post by kevingadd »

cjbgkagh wrote: post #7
TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

I have similar thoughts, I don't understand the push for variable width SIMD. Possibly due to ignorance but I think it's an abstraction that can be specialized for different hardware so the similar tradeoffs between low level languages and high level languages apply. Since I already have to be aware of hardware level concepts such as 256bit shuffle not working across 128bit lanes and different instructions having ver…

Part of the motive behind variable width SIMD in WASM is that there's intentionally-ish no mechanism to do feature detection at runtime in WASM. The whole module has to be valid on your target, you can't include a handful of invalid functions and conditionally execute them if the target supports 256-wide or 512-wide SIMD. If you want to adapt you have to ship entire modules for each set of supported feature flags and select the correct module at startup after probing what the target supports.

So variable width SIMD solves this by making any module using it valid regardless of whether the target supports 512-bit vectors, and the VM 'just' has to solve the problem of generating good code.

Personally I think this is a terrible way to do things and there should have just been a feature detection system, but the horse fled the barn on that one like a decade ago.

Re: Fundamental flaws of SIMD ISAs (2021)

#35

Post by convolvatron »

adgjlsfhk1 wrote: post #30
convolvatron wrote: post #3

i would certainly add lack of reductions ('horizontal' operations) and a more generalized model of communication to the list.

The tricky part with reductions is that they are somewhat inherently slow since they often need to be done pairwise and a pairwise reduction over 16 elements will naturally have pretty limited parallelism.

kinda? this is sort of a direct result of the 'vectors are just sliced registers' model. if i do a pairwise operation and divide my domain by 2 at each step, is the resulting vector sparse or dense? if its dense then I only really top out when i'm in the last log2slice steps.

Re: Fundamental flaws of SIMD ISAs (2021)

#36

Post by camel-cdr »

pkhuong wrote: post #21
camel-cdr wrote: post #12

Earlier quoted context omitted.

> I prefer fixed width Do you have examples for problems that are easier to solve in fixed-width SIMD? I maintain that most problems can be solved in a vector-length-agnostic manner. Even if it's slightly more tricky, it's certainly easier than restructuring all of your memory allocations to add padding and implementing three versions for all the differently sized SIMD extensions your target may support. And you can…

> Do you have examples for problems that are easier to solve in fixed-width SIMD? Regular expression matching and encryption come to mind.

> Regular expression matching

That's probably true. Last time I looked at it, it seemed like parts of vectorscan could be vectorized VLA, but from my, very limited, understanding of the main matching algorithm, it does seem to require specialization on vector length.

It should be possible to do VLA in some capacity, but it would probably be slower and it's too much work to test.

> encryption

From the things I've looked at, it's mixed.

E.g. chacha20 and poly1305 vectorize well in a VLA scheme: https://camel-cdr.github.io/rvv-bench-results/bpi_f3/chacha2..., https://camel-cdr.github.io/rvv-bench-results/bpi_f3/poly130...

Keccak on the other hand was optimized for fast execution on scalar ISAs with 32 GPRs. This is hard to vectorize in general, because GPR "moves" are free and liberally applied.

Another example where it's probably worth specializing is quicksort, specifically the leaf part.

I've written a VLA version, which uses bitonic sort to sort within vector registers. I wasn't able to meaningfully compare it against a fixed size implementation, because vqsort was super slow when I tried to compile it for RVV.

Re: Fundamental flaws of SIMD ISAs (2021)

#37

Post by camel-cdr »

timewizard wrote: post #33

> Another problem is that each new SIMD generation requires new instruction opcodes and encodings. It requires new opcodes. It does not strictly require new encodings. Several new encodings are legacy compatible and can encode previous generations vector instructions. > so the architecture must provide enough SIMD registers to avoid register spilling. Or the architecture allows memory operands. The great joy of basic…

> The great joy of basic x86 encoding is that you don't actually need to put things in registers to operate on them.

That's just spilling with fewer steps. The executed uops should be the same.

Re: Fundamental flaws of SIMD ISAs (2021)

#38

Post by Remnant44 »

sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

These days, I strongly believe that loop unrolling is a pessimization, especially with SIMD code.

Scalar code should be unrolled by the compiler to the SIMD word width to expose potential parallelism. But other than that, correctly predicted branches are free, and so is loop instruction overhead on modern wide-dispatch processors. For example, even running a maximally efficient AVX512 kernel on a zen5 machine that dispatches 4 EUs and some load/stores and calculates 2048 bits in the vector units every cycle, you still have a ton of dispatch capacity to handle the loop overhead in the scalar units.

The cost of unrolling is decreased code density and reduced effectiveness of the instruction / uOp cache. I wish Clang in particular would stop unrolling the dang vector loops.

Re: Fundamental flaws of SIMD ISAs (2021)

#39

Post by luyu_wu »

tonetegeatinst wrote: post #26
TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

AFAIK about every modern CPU uses out of order von Neumann architecture. The only people who don't are the handful of researchers and people who work with the government research into non van Neumann designed systems.

Low power RISC cores (both ARM and RISC-V) are typically in-order actually!

But any core I can think of as 'high-performance' is OOO.

Re: Fundamental flaws of SIMD ISAs (2021)

#40

Post by adgjlsfhk1 »

Remnant44 wrote: post #38
sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

These days, I strongly believe that loop unrolling is a pessimization, especially with SIMD code. Scalar code should be unrolled by the compiler to the SIMD word width to expose potential parallelism. But other than that, correctly predicted branches are free, and so is loop instruction overhead on modern wide-dispatch processors. For example, even running a maximally efficient AVX512 kernel on a zen5 machine that di…

The part that's really weird is that on modern CPUs predicted branches are free iff they're sufficiently rare (<1 out of 8 instructions or so). but if you have too many, you will be bottlenecked on the branch since you aren't allowed to speculate past a 2nd (3rd on zen5 without hyperthreading?) branch.
Post reply on HN