Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fundamental flaws of SIMD ISAs (2021)

bitsnbites.eu

Post reply on HN
Print view View original

111–120 of 146 posts

Re: Fundamental flaws of SIMD ISAs (2021)

#111

Post by Someone »

> Since the register size is fixed there is no way to scale the ISA to new levels of hardware parallelism without adding new instructions and registers.

I think there is a way: vary register size per CPU, but also add an instruction to retrieve register size. Then, code using the vector unit will sometimes have to dynamically allocate a buffer for intermediate values, but it would allow for software to run across CPUs with different vector lengths. Does anybody know whether any architecture does this?

Re: Fundamental flaws of SIMD ISAs (2021)

#112

Post by almostgotcaught »

pornel wrote: post #105
almostgotcaught wrote: post #80

Earlier quoted context omitted.

> There are alternative universes where these wouldn't be a problem Do people that say these things have literally any experience of merit? > For example, if we didn't settle on executing compiled machine code exactly as-is, and had a instruction-updating pass You do understand that at the end of the day, hardware is hard (fixed) and software is soft (malleable) right? There will be always be friction at some boundar…

The point of updating the instructions isn't to have optimal behavior in all cases, or to reconfigure programs for wildly different hardware, but to be able to easily target contemporary hardware, without having to wait for the oldest hardware to die out first to be able to target a less outdated baseline without conditional dispatch. Users are much more forgiving about software that runs a bit slower than software t…

i have no idea what you're saying - i'm well aware that compilers do lots of things but this sentence in your original comment

> compiled machine code exactly as-is, and had a instruction-updating pass

implies there should be silicon that implements the instruction-updating - what else would be "executing" compiled machine code other than the machine itself...........

Re: Fundamental flaws of SIMD ISAs (2021)

#113

Post by the__alchemist »

PaulHoule wrote: post #28
TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

In AVX-512 we have a platform that rewards the assembly language programmer like few platforms have since the 6502. I see people doing really clever things that are specific to the system and one level it is really cool but on another level it means SIMD is the domain of the specialist, Intel puts out press releases about the really great features they have for the national labs and for Facebook whereas the rest of u…

Noob question! What about AVX-512 makes it unique to assembly programmers? I'm just dipping my toes in, and have been doing some chemistry computations using f32x8, Vec3x8 etc (AVX-256). I have good workflows set up, but have only been getting 2x speedup over non-SIMD. (Was hoping for closer to 8). I figured AVX-512 would allow f32x16 etc, which would be mostly a drop-in. (I have macros to set up the types, and you input num lanes).

Re: Fundamental flaws of SIMD ISAs (2021)

#114

Post by kjs3 »

Const-me wrote: post #107
TinkersW wrote: post #4

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as t…

> I prefer fixed width Another reason to prefer fixed width, compilers may pass vectors to functions in SIMD registers. When register size is unknown at compile time, they have to pass data in memory. For complicated SIMD algorithms the performance overhead gonna be huge.

Back in the day, you had Cray style vector registers, and you had CDC style[1] 'vector pipes' (I think I remember that's what they called them) that you fed from main memory. So you would (vastly oversimplifying) build your vectors in consecutive memory locations (up to 64k as I recall), point to a result destination in memory and execute a vector instruction. This works fine if there's a close match between cpu speed and memory access speed. The compilers were quite good, and took care of handling variable sized vectors, but I have no idea what was going on under the hood except for some hi-level undergrad compiler lectures. As memory speed vs cpu speed divergence became more and more pronouced, it quickly became obvious that vector registers were the right performance answer, basically everyone jumped that way, and I don't think anyone has adopted a memory-memory vector architecture since the '80s.

[1] from CDC STAR-100 and followons like the CDC Cyber 180/990, Cyber 200 series & ETA-10.

Re: Fundamental flaws of SIMD ISAs (2021)

#115

Post by dzaima »

dragontamer wrote: post #72
dzaima wrote: post #70

Earlier quoted context omitted.

That's two instrs - bumping the indices, and doing the comparison. You still want scalar pointer/index bumping for contiguous loads/stores (using gathers/scatters for those would be stupid and slow), and that gets you the end check for free via fused cmp+jcc. And those two instrs are vector instrs, i.e. competing with execution units for the actual thing you want to compute, whereas scalar instrs have at least some i…

Gotcha. Makes sense. Thanks for the discussion! Overall, I agree that AVX and Neon have their warts and performance issues. But they're like 15+ years old now and designed well before GPU Compute was possible. > using gathers/scatters for those would be stupid and slow This is where CPUs are really bad. GPUs will coalesce gather/scatters thanks to __shared__ memory (with human assistance of course). But also the simp…

Thanks for the info on how things look on the GPU side!

A messy thing with memory performance on CPUs is that either you share the same cache hardware between scalar and vector, thereby significantly limiting how much latency you can trade for throughput, or you have to add special vector L1 cache, which is a ton of mess and silicon area; never mind uses of SIMD that are latency-sensitive, e.g. SIMD hashmap probing, or small loops.

I guess you don't necessarily need that for just detecting patterns in gather indices, but nothing's gonna get a gather of consecutive 8-bit elts via 64-bit indices to not perform much slower than a single contiguous load, and 8-bit elts are quite important on CPUs for strings & co.

Re: Fundamental flaws of SIMD ISAs (2021)

#116

Post by LoganDark »

codedokode wrote: post #93
LoganDark wrote: post #91

Earlier quoted context omitted.

This comment sort of reminds me of how Transmeta CPUs relied on the compiler to precompute everything like pipelining. It wasn't done by the hardware.

Makes sense - writing or updating software is easier that designing or updating hardware. To illustrate: anyone can write software but not everyone has access to chip manufacturing fabs.

Atomic Semi may be looking to change that (...eventually)

Re: Fundamental flaws of SIMD ISAs (2021)

#117

Post by imtringued »

sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

Ok, but the compiler can't do that without unrolling.

Re: Fundamental flaws of SIMD ISAs (2021)

#118

Post by pornel »

almostgotcaught wrote: post #112
pornel wrote: post #105

Earlier quoted context omitted.

The point of updating the instructions isn't to have optimal behavior in all cases, or to reconfigure programs for wildly different hardware, but to be able to easily target contemporary hardware, without having to wait for the oldest hardware to die out first to be able to target a less outdated baseline without conditional dispatch. Users are much more forgiving about software that runs a bit slower than software t…

i have no idea what you're saying - i'm well aware that compilers do lots of things but this sentence in your original comment > compiled machine code exactly as-is, and had a instruction-updating pass implies there should be silicon that implements the instruction-updating - what else would be "executing" compiled machine code other than the machine itself...........

I was talking about a software pass. Currently, the machine code stored in executables (such as ELF or PE) is only slightly patched by the dynamic linker, and then expected to be directly executable by the CPU. The code in the file has to be already compatible with the target CPU, otherwise you hit illegal instructions. This is a simplistic approach, dating back to when running executables was just a matter of loading them into RAM and jumping to their start (old a.out or DOS COM).

What I'm suggesting is adding a translation/fixup step after loading a binary, before the code is executed, to make it more tolerant to hardware changes. It doesn’t have to be full abstract portable bytecode compilation, and not even as involved as PTX to SASS, but more like a peephole optimizer for the same OS on the same general CPU architecture. For example, on a pre-AVX2 x86_64 CPU, the OS could scan for AVX2 instructions and patch them to do equivalent work using SSE or scalar instructions. There are implementation and compatibility issues that make it tricky, but fundamentally it should be possible. Wilder things like x86_64 to aarch64 translation have been done, so let's do it for x86_64-v4 to x86_64-v1 too.

Re: Fundamental flaws of SIMD ISAs (2021)

#119

Post by p_l »

whaleofatw2022 wrote: post #44
luyu_wu wrote: post #39

Earlier quoted context omitted.

Low power RISC cores (both ARM and RISC-V) are typically in-order actually! But any core I can think of as 'high-performance' is OOO.

MIPS as well as Alpha AFAIR. And technically itanium, otoh It seems to me a bit like a niche for any performance advantages...

Alpha was out of order starting with EV7, but most importantly the entire architecture was designed with eye for both pipeline hazards and out of order execution, unlike VAX that it replaced which made it pretty much impossible

Re: Fundamental flaws of SIMD ISAs (2021)

#120

Post by dzaima »

the__alchemist wrote: post #113
PaulHoule wrote: post #28

Earlier quoted context omitted.

In AVX-512 we have a platform that rewards the assembly language programmer like few platforms have since the 6502. I see people doing really clever things that are specific to the system and one level it is really cool but on another level it means SIMD is the domain of the specialist, Intel puts out press releases about the really great features they have for the national labs and for Facebook whereas the rest of u…

Noob question! What about AVX-512 makes it unique to assembly programmers? I'm just dipping my toes in, and have been doing some chemistry computations using f32x8, Vec3x8 etc (AVX-256). I have good workflows set up, but have only been getting 2x speedup over non-SIMD. (Was hoping for closer to 8). I figured AVX-512 would allow f32x16 etc, which would be mostly a drop-in. (I have macros to set up the types, and you i…

SIMD only helps you where you're arithmetic-limited; you may be limited by memory bandwidth, or perhaps float division if applicable; and if your scalar comparison got autovectorized you'd have roughly no benefit.

AVX-512 should be just fine via intrinsics/high-level vector types, not different from AVX2 in this regard.

Post reply on HN