Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fundamental flaws of SIMD ISAs (2021)

bitsnbites.eu

Post reply on HN
Print view View original

91–100 of 146 posts

Re: Fundamental flaws of SIMD ISAs (2021)

#91

Post by LoganDark »

codedokode wrote: post #83

I think that packed SIMD is better in almost every aspect and Vector SIMD is worse. With vector SIMD you don't know the register size beforehand and therefore have to maintain and increment counters, adding extra unnecessary instructions, reducing total performance. With packed SIMD you can issue several loads immediately without dependencies, and if you look at code examples, you can see that the x86 code is more de…

This comment sort of reminds me of how Transmeta CPUs relied on the compiler to precompute everything like pipelining. It wasn't done by the hardware.

Re: Fundamental flaws of SIMD ISAs (2021)

#92

Post by codedokode »

sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

> Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four.

I cannot agree because in an unrolled loop you have less counter increment instructions.

Re: Fundamental flaws of SIMD ISAs (2021)

#93

Post by codedokode »

LoganDark wrote: post #91
codedokode wrote: post #83

I think that packed SIMD is better in almost every aspect and Vector SIMD is worse. With vector SIMD you don't know the register size beforehand and therefore have to maintain and increment counters, adding extra unnecessary instructions, reducing total performance. With packed SIMD you can issue several loads immediately without dependencies, and if you look at code examples, you can see that the x86 code is more de…

This comment sort of reminds me of how Transmeta CPUs relied on the compiler to precompute everything like pipelining. It wasn't done by the hardware.

Makes sense - writing or updating software is easier that designing or updating hardware. To illustrate: anyone can write software but not everyone has access to chip manufacturing fabs.

Re: Fundamental flaws of SIMD ISAs (2021)

#94

Post by immibis »

PaulHoule wrote: post #52
bee_rider wrote: post #50

Earlier quoted context omitted.

It is kind of a bummer that MKL isn’t open sourced, as that would make inclusion in Linux easier. It is already free-as-in-beer, but of course that doesn’t solve everything. Baffling that MS didn’t use it. They have a pretty close relationship… Agree that they are sort of going after hard-to-use niche features nowadays. But I think it is just that the real thing we want—single threaded performance for branchy code—is…

At the very least you can decode UTF-8 really quickly with AVX-512 https://lemire.me/blog/2023/08/12/transcoding-utf-8-strings-... and web browsers at the very least spent a lot of cycles on decoding HTML and Javascript which is UTF-8 encoded. It turns out AVX-512 is good at a lot of things you wouldn't think SIMD would be good at. Intel's got the problem that people don't want to buy new computers because they don't…

If you pay attention this isn't a UTF-8 decoder. It might be some other encoding, or a complete misunderstanding of how UTF-8 works, or an AI hallucination. It also doesn't talk about how to handle the variable number of output bytes or the possibility of a continuation sequence split between input chunks.

Re: Fundamental flaws of SIMD ISAs (2021)

#95

Post by ezekiel68 »

ack_complete wrote: post #66
PaulHoule wrote: post #28

Earlier quoted context omitted.

In AVX-512 we have a platform that rewards the assembly language programmer like few platforms have since the 6502. I see people doing really clever things that are specific to the system and one level it is really cool but on another level it means SIMD is the domain of the specialist, Intel puts out press releases about the really great features they have for the national labs and for Facebook whereas the rest of u…

AVX-512 also has a lot of wonderful facilities for autovectorization, but I suspect its initial downclocking effects plus getting yanked out of Alder Lake killed a lot of the momentum in improving compiler and library usage of it. Even the Steam Hardware Survey, which is skewed toward upper end hardware, only shows 16% availability of baseline AVX-512, compared to 94% for AVX2.

You mentioned "initial downclocking effects", yet (for posterity) I want to emphasize that in 2020 Ice Lake (Sunny Cove core) and later Intel processors, the downclocking is really a nothingburger. The fusing off debacle in desktop CPU families like Alder Lake you mentioned definitely killed the momentum though.

I'm not sure why OS kernels couldn't have become partners in CPU capability queries (where a program starting execution could request a CPU core with 'X' such as AVX-512F, for example) -- but without that the whole P-core/E-core hybrid concept was DOA for capabilities which were not least-common denominators. If I had to guess, marketing got ahead of engineering and testing on that one.

Re: Fundamental flaws of SIMD ISAs (2021)

#96

Post by Zambyte »

codedokode wrote: post #85
derf_ wrote: post #49

Earlier quoted context omitted.

> I code with SIMD as the target, and have special containers that pad memory to SIMD width... I think this may be domain-specific. I help maintain several open-source audio libraries, and wind up being the one to review the patches when people contribute SIMD for some specific ISA, and I think without exception they always get the tail handling wrong. Due to other interactions it cannot always be avoided by padding.…

I think that SIMD code should not be written by hand but rather in a high-level language and so dealing with tail becomes a compiler's and not a programmer's problem. Or people still prefer to write assembly be hand? It seems to be so judging by the code you link. What I wanted is to write code in a more high-level language like this. For example, to compute a scalar product of a and b you write: 1..n | a[$1] * b[$1]…

Zig exposes a Vector type to use for SIMD instructions, which has been my first introduction to SIMD directly. Reading through this thread I was immediately mapping what people were saying to Vector operations in Zig. It seems to me like SIMD can reasonably be exposed in high level languages for programmers to reach to in contexts where it matters.

Of course, the compiler vectorizing code when it can as a general optimization is still useful, but when it's critical that some operations must be vectorized, explicit SIMD structures seem nice to have.

Re: Fundamental flaws of SIMD ISAs (2021)

#97

Post by whizzter »

adgjlsfhk1 wrote: post #81
ack_complete wrote: post #73

Earlier quoted context omitted.

No proof, but I suspect that AMD's AVX-512 support played a part in Intel dumping AVX10/256 and changing plans back to shipping a full 512-bit consumer implementation again (we'll see when they actually ship it). The downside is that AMD also increased the latency of all formerly cheap integer vector ops. This removes one of the main advantages against NEON, which historically has had richer operations but worse late…

> For instance, Skymont finally has no penalty for denormals, a long standing Intel weakness. yeah, that's crazy to me. Intel has been so completely discunctional for the last 15 years. I feel like you couldn't have a clearer sign of "we have 2 completely separate teams that are competing with each other and aren't allowed to/don't want to talk to each other". it's just such a clear sign that the chicken is running a…

Not really, to me it more seems like Pentium-4 vs Pentium-M/Core again.

The downfall of Pentium 4 was that they had been stuffing things into longer and longer pipes to keep up the frequency race(with horrible branch latencies as a result). They scaled it all away by "resetting" to the P3/P-M/Core architecture and scaling up from that again.

Pipes today are even _longer_ and if E-cores has shorter pipes at a similar frequency then "regular" JS,Java,etc code will be far more performant even if you lose a bit of perf for "performance" cases where people vectorize (Did the HPC computing crowd influence Intel into a ditch AGAIN? wouldn't be surprising!).

Re: Fundamental flaws of SIMD ISAs (2021)

#98

Post by pornel »

LegionMammal978 wrote: post #9
pornel wrote: post #6

There are alternative universes where these wouldn't be a problem. For example, if we didn't settle on executing compiled machine code exactly as-is, and had a instruction-updating pass (less involved than a full VM byte code compilation), then we could adjust SIMD width for existing binaries instead of waiting decades for a new baseline or multiversioning faff. Another interesting alternative is SIMT. Instead of hav…

> Another interesting alternative is SIMT. Instead of having a handful of special-case instructions combined with heavyweight software-switched threads, we could have had every instruction SIMDified. It requires structuring programs differently, but getting max performance out of current CPUs already requires SIMD + multicore + predictable branching, so we're doing it anyway, just in a roundabout way. Is that not whe…

CPUs are not good at branchy code either. Branch mispredictions cause costly pipeline stalls, so you have to make branches either predictable or use conditional moves. Trivially predictable branches are fast — but so are non-diverging warps on GPUs. Conditional moves and masked SIMD work pretty much exactly like on a GPU.

Even if you have a branchy divide-and-conquer problem ideal for diverging threads, you'll get hit by a relatively high overhead of distributing work across threads, false sharing, and stalls from cache misses.

My hot take is that GPUs will get more features to work better on traditionally-CPU-problems (e.g. AMD Shader Call proposal that helps processing unbalanced tree-structured data), and CPUs will be downgraded to being just a coprocessor for bootstrapping the GPU drivers.

Re: Fundamental flaws of SIMD ISAs (2021)

#99

Post by gpderetta »

sweetjuly wrote: post #10

Loop unrolling isn't really done because of pipelining but rather to amortize the cost of looping. Any modern out-of-order core will (on the happy path) schedule the operations identically whether you did one copy per loop or four. The only difference is the number of branches.

The looping overhead is trivial, especially on simd code where the loop overhead will use the scalar hardware.

Unrolling is definitely needed for properly scheduling and pipelining SIMD code even on OoO cores. Remember that an OoO core cannot reorder dependent instructions, so the dependencies need to be manually broken, for example by adding additional accumulators, which in turn requires additional unrolling, this is especially important on SIMD code which typically is non-branchy with long dependency chains.

Re: Fundamental flaws of SIMD ISAs (2021)

#100

Post by gpderetta »

dzaima wrote: post #43
adgjlsfhk1 wrote: post #40

Earlier quoted context omitted.

The part that's really weird is that on modern CPUs predicted branches are free iff they're sufficiently rare (<1 out of 8 instructions or so). but if you have too many, you will be bottlenecked on the branch since you aren't allowed to speculate past a 2nd (3rd on zen5 without hyperthreading?) branch.

The limiting thing isn't necessarily speculating, but more just the number of branches per cycle, i.e. number of non-contiguous locations the processor has to query from L1 / uop cache (and which the branch predictor has to determine the location of). You get that limit with unconditional branches too.

Indeed, the limit is on taken branches, hence why making the most likely case fall through is often an optimization.
Post reply on HN