Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Execution units are often pipelined

blog.xoria.org

Post reply on HN
Print view View original

61–70 of 102 posts

Re: Execution units are often pipelined

#61

Post by gpderetta »

wtallis wrote: post #57
gpderetta wrote: post #51

Earlier quoted context omitted.

> and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution Being able to execute multiple instructions is more properly superscalar execution, right? In-order designs are also capable of doing it and the separate execution unit do not even need to run in lockstep (consider the origi…

Right; it's easy to forget that superscalar CPU cores don't actually have to be in-order, but most of them are out-of-order because that's usually necessary to make good use of a wide superscalar core. (What's the best-performing in-order general purpose CPU core? POWER6 was notably in-order and ran at quite high clock speeds for the time. Intel's first-gen Atom cores were in-order and around the same time as POWER6…

POWER6 might indeed have been the last In-Order speed demon.

Re: Execution units are often pipelined

#62

Post by gpderetta »

dhash wrote: post #59

My favorite illustrations for the concepts discussed here (in an accessible form, not the processor optimization manuals) has long been [0]. For me, this really makes working with a modern microprocessor a science , as anyone who has written benchmarks knows -- it's difficult to reason about the complex behaviour and performance cliffs without testing. Another excellent example of the weirdness has to be JVM anatomy…

The first link is very nice, worth of a submission of its own.

Re: Execution units are often pipelined

#63

Post by thesz »

bonzini wrote: post #42
filiphorvat wrote: post #41

Unrelated, do FPUs on modern CPUs use FMAs to both multiply and add or do they use mul/add-only units?

Probably to do multiplies, as the extra add is basically free. Adds are cheaper.

Adds are cheaper only for fixed-point computations. Floating point addition needs to denormalize one of its' arguments, perform an (integer) addition and then normalize the result.

Usually FP adds take a cycle or two longer than FP multiplication.

Re: Execution units are often pipelined

#64

Post by ajross »

gpderetta wrote: post #60
ajross wrote: post #52

Earlier quoted context omitted.

In-order parallel designs are "VLIW". The jargon indeed gets thick. :) But as to OO: the whole idea of issuing sequential instructions in parallel means that the hardware needs to track dependencies between them so they can't race ahead of their inputs. And if you're going to do that anyway, allowing them to retire out of order is a big performance/transistor-count win as it allows the pipeline lengths to be differen…

VLIW is again a different thing. It uses a single instruction that encodes multiple independent operations to simplify decoding and tracking, usually with exposed pipelines. But you can have, for example, a classic in-order RISC design that allows for parallel execution. OoO renaming is not necessary for dependency tracking (in fact even scalar in order CPUs need dependency tracking to solve RAW and other hazards), i…

> VLIW is again a different thing.

No it isn't. I'm being very deliberate here with refusing pedantry. In practice, "multiple dispatch" means "OO" in the same way that "VLIW" means "parallel in order dispatch". Yes, you can imagine hypothetical CPUs that mix the distinction, but they'd be so weird that they'd never be built. Discussing the jargon without context only confuses things.

> you can have, for example, a classic in-order RISC design that allows for parallel execution.

Only by inventing VLIW, though, otherwise there's no way to tell the CPU how to order what it does. Which is my point; the ideas are joined at the hip. Note that the Pentium had two defined pipes with specific rules about how the pairing was encoded in the instruction stream. It was, in practice, a VLIW architecture (just one with a variable length encoding and where most of the available instruction bundles only filled one slot)! Pedantry hurts in this world, it doesn't help.

Re: Execution units are often pipelined

#65

Post by gpderetta »

filiphorvat wrote: post #41

Unrelated, do FPUs on modern CPUs use FMAs to both multiply and add or do they use mul/add-only units?

I don't think there is a generally optimal design. There are cons and pros to using the same homogeneous FMAs units for adds, multiplies and fmas, even at the cost of making adds slower (simpler design, and having all instructions of the same latency greatly simplifies scheduling). IIRC intel cycled through 4 cycles fma, add and mul, then to 4 cycles add and mul and 5 cycles fmas, then with a dedicated 3 cycles add.

The optimal design depends a lot on the rest of the microarchitecture, the loads the core is being optimized for, the target frequency, the memory latency, etc.

Re: Execution units are often pipelined

#66

Post by Tuna-Fish »

delusional wrote: post #3

Is this still the case if I have different ALU operations. Say I have a single ALU on a single x86 core. Would the ALU be able to interleave say ADD and MULs? or would I incur the latency measure for each operation switch? I know that some ALU's have multiple ADD complexes, and I assume that would influence the answer, hence why I specified x86.

You can interleave most operations how you like, without any extra latency, each op starting as soon as all the results are ready, regardless of where they were computed.

There are some exceptions where "domain crossing", or using a very different operation costs an extra clock cycle. Notably, in vector registers using FP or integer operations on the result of the different type of op, as modern CPUs don't actually hold FP values in their IEEE754 transfer format inside registers, but instead registers have hidden extra bits and store all values in normal form, allowing fast operations on denormals. The downside of this is that if you alternate between FP and INT operations, the CPU has to insert extra conversion ops between them. Typical cost is 1-3 cycles per domain crossing.

Re: Execution units are often pipelined

#67

Post by eigenform »

ajross wrote: post #17
Vogtinator wrote: post #2

For x86 cores this is visible in Agner Fog's instruction performance tables: https://agner.org/optimize/#manuals The latency shows after how many cycles the result of an instruction can be consumed by another, while the throughput shows how many such instructions can be pipelined per cycle, i.e. in parallel.

FWIW, there are two ideas of parallelism being conflated here. One is the parallel execution of the different sequential steps of an instruction (e.g. fetch, decode, operate, retire). That's "pipelining", and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution" tends to connote the…

> Also IIRC there are still some non-pipelined units in Intel chips, like the division engine, which show latency numbers ~= to their execution time

I don't think that's accurate. That latency exists because the execution unit is pipelined. If it were not pipelined, there would be no latency. The latency corresponds to the fact that "doing division" is distributed across multiple clock cycles.

Re: Execution units are often pipelined

#68

Post by gpderetta »

ajross wrote: post #64
gpderetta wrote: post #60

Earlier quoted context omitted.

VLIW is again a different thing. It uses a single instruction that encodes multiple independent operations to simplify decoding and tracking, usually with exposed pipelines. But you can have, for example, a classic in-order RISC design that allows for parallel execution. OoO renaming is not necessary for dependency tracking (in fact even scalar in order CPUs need dependency tracking to solve RAW and other hazards), i…

> VLIW is again a different thing. No it isn't. I'm being very deliberate here with refusing pedantry. In practice , "multiple dispatch" means "OO" in the same way that "VLIW" means "parallel in order dispatch". Yes, you can imagine hypothetical CPUs that mix the distinction, but they'd be so weird that they'd never be built. Discussing the jargon without context only confuses things. > you can have, for example, a c…

I'm sorry, but if P5 was VLIW then the word has lost all meanings. They couldn't possibly be more different.

Re: Execution units are often pipelined

#69

Post by BeeOnRope »

Earw0rm wrote: post #33
BeeOnRope wrote: post #16

Earlier quoted context omitted.

That's true, but another part of the tables show how many "ports" the operation can be executed on, which is enough information to concluded an operation is pipelined. For example, for many years Intel chips had a multiplier unit on a single port, with a latency of 3 cycles, but an inverse throughput of 1 cycle, so effectively pipelined across 3 stages. In any case, I think uops.info [1] has replaced Agner for up-to-…

Shame it doesn't seem to have been updated with Arrow Lake, Zen 5 and so on yet.

Yes. In the past new HW has been made available to the uops.info authors in order to run their benchmark suite and publish new numbers: I'm not sure if that just hasn't happened for the new stuff, or if they are not interested in updating it.

Re: Execution units are often pipelined

#70

Post by BeeOnRope »

ajross wrote: post #38
BeeOnRope wrote: post #18

Earlier quoted context omitted.

I don't think anyone is talking about "fetch, decode, operate, retire" pipelining (though that is certainly called pipelinig): only pipelining within the execution of a instruction that takes multiple cycles just to execute (i.e., latency from input-ready to output-ready). Pipelining in stages like fetch and decode are mostly hidden in these small benchmarks, but are visible when there are branch misprediction, other…

> I don't think anyone is talking about "fetch, decode, operate, retire" pipelining (though that is certainly called pipelinig): only pipelining within the execution of a instruction that takes multiple cycles just to execute (i.e., latency from input-ready to output-ready). I'm curious what you think the distinction is? Those statements are equivalent. The circuit implementing "an instruction" can't work in a single…

We are interested in the software visible performance effects of pipelining. For small benchmarks that don't miss in the predictors or icache, this mostly means execution pipelining. That's the type of pipelining the article is discussing and the type of pipelining considered in instruction performance breakdowns considered by Agner, uops.info, simulated by LLVM-MCA, etc.

I.e., a lot of what you need to model for tight loops only depends on the execution latencies (as little as 1 cycle), and not on the full pipeline end-to-end latency (almost always more than 10 cycles on big OoO, maybe more than 20).

Post reply on HN