Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Execution units are often pipelined

blog.xoria.org

Post reply on HN
Print view View original

51–60 of 102 posts

Re: Execution units are often pipelined

#51

Post by gpderetta »

ajross wrote: post #17
Vogtinator wrote: post #2

For x86 cores this is visible in Agner Fog's instruction performance tables: https://agner.org/optimize/#manuals The latency shows after how many cycles the result of an instruction can be consumed by another, while the throughput shows how many such instructions can be pipelined per cycle, i.e. in parallel.

FWIW, there are two ideas of parallelism being conflated here. One is the parallel execution of the different sequential steps of an instruction (e.g. fetch, decode, operate, retire). That's "pipelining", and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution" tends to connote the…

> and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution

Being able to execute multiple instructions is more properly superscalar execution, right? In-order designs are also capable of doing it and the separate execution unit do not even need to run in lockstep (consider the original P5 U and V pipes).

Re: Execution units are often pipelined

#52

Post by ajross »

gpderetta wrote: post #51
ajross wrote: post #17

Earlier quoted context omitted.

FWIW, there are two ideas of parallelism being conflated here. One is the parallel execution of the different sequential steps of an instruction (e.g. fetch, decode, operate, retire). That's "pipelining", and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution" tends to connote the…

> and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution Being able to execute multiple instructions is more properly superscalar execution, right? In-order designs are also capable of doing it and the separate execution unit do not even need to run in lockstep (consider the origi…

In-order parallel designs are "VLIW". The jargon indeed gets thick. :)

But as to OO: the whole idea of issuing sequential instructions in parallel means that the hardware needs to track dependencies between them so they can't race ahead of their inputs. And if you're going to do that anyway, allowing them to retire out of order is a big performance/transistor-count win as it allows the pipeline lengths to be different.

Re: Execution units are often pipelined

#53

Post by LegionMammal978 »

devit wrote: post #49
alain94040 wrote: post #32

Earlier quoted context omitted.

Very true. To paraphrase a saying, CPU amateurs argue about micro-benchmarks on HN, the pros simulate real code.

The amateurs usually run benchmarks (because they can't reason about it as they lack the relevant knowledge) and believe they got a useful result on some aspect when in the fact the benchmark usually depends on other arbitrary random factors (e.g. maybe they think they are measuring FMA throughput, but are in fact measuring whether the compiler autovectorizes or whether it fuses multiply and adds automatically). A pr…

Then again, 'reasoning about it' can easily go awry if your knowledge doesn't get updated alongside the CPU architectures. I've seen people confidently say all sorts of stuff about optimization that hasn't been relevant since the early 2000s. Or in the other direction, some people treat modern compilers/CPUs like they can perform magic, so that it makes no difference what you shovel into them (e.g., "this OOP language has a compiler that always knows when to store values on the stack"). Benchmarks can help dispel some of the most egregious myths, even if they are easy to misuse.

Re: Execution units are often pipelined

#54

Post by Joker_vD »

barbegal wrote: post #4

These days CPUs are so complex and have so many interdependencies that the best way to simulate them is simply to run them! In most real code the high throughput of these sorts of operations means that something else is the limiting factor. And if multiplier throughput is limiting performance then you should be using SIMD or a GPU.

> the best way to simulate them is simply to run them!

And it's quite sad because when you are faced with choosing between two ways to express something in the code, you can't predict how fast one or another option will run. You need to actually run both, preferrably in an environment close to the prod, and under similar load, to get accurate idea which one is more performant.

And the worst thing is, you most likely can't extract any useful general principle out of it, because any small perturbation in the problem will result in a code that is very similar yet has completely different latency/throughput characteristics.

The only saving grace is that modern computers are really incredibly fast, so layers upon layers of suboptimal code result in applications that mostly perform okay, with maybe some places where they perform egregiously slow.

Re: Execution units are often pipelined

#55

Post by gpderetta »

gary_0 wrote: post #48
rasz wrote: post #43

Pipelining is the direct reason behind 1996 Quake running 30 fps on Intel Pentium 133 but requiring 233MHz from Cyrix and 166MHz from AMD K6 to reach same result. 20 fps needed 75MHz Intel Pentium, 133MHz Cyrix and 100MHz AMD K5.

Specifically, Carmack exploited the fact that on the Pentium, integer instructions could run in parallel with floating-point division[0]. This goes to show that an optimization that usually gets you ~10% might get you 200% depending on what software you're running. And that no implementation detail is safe from an ambitious software engineer. [0] https://news.ycombinator.com/item?id=38249029

So that was not actually pipelining, but superscalar execution. Incidentally division wasn't pipelined at all.

Re: Execution units are often pipelined

#57

Post by wtallis »

gpderetta wrote: post #51
ajross wrote: post #17

Earlier quoted context omitted.

FWIW, there are two ideas of parallelism being conflated here. One is the parallel execution of the different sequential steps of an instruction (e.g. fetch, decode, operate, retire). That's "pipelining", and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution" tends to connote the…

> and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution Being able to execute multiple instructions is more properly superscalar execution, right? In-order designs are also capable of doing it and the separate execution unit do not even need to run in lockstep (consider the origi…

Right; it's easy to forget that superscalar CPU cores don't actually have to be in-order, but most of them are out-of-order because that's usually necessary to make good use of a wide superscalar core.

(What's the best-performing in-order general purpose CPU core? POWER6 was notably in-order and ran at quite high clock speeds for the time. Intel's first-gen Atom cores were in-order and around the same time as POWER6 but at half the clock speed. SPARC T3 was ran at an even lower clock speed.)

Re: Execution units are often pipelined

#58

Post by mpweiher »

Joker_vD wrote: post #54
barbegal wrote: post #4

These days CPUs are so complex and have so many interdependencies that the best way to simulate them is simply to run them! In most real code the high throughput of these sorts of operations means that something else is the limiting factor. And if multiplier throughput is limiting performance then you should be using SIMD or a GPU.

> the best way to simulate them is simply to run them! And it's quite sad because when you are faced with choosing between two ways to express something in the code, you can't predict how fast one or another option will run. You need to actually run both, preferrably in an environment close to the prod, and under similar load, to get accurate idea which one is more performant. And the worst thing is, you most likely…

> you can't predict how fast one or another option will run

"The best way to predict the future is to invent it" -- Alan Kay

> You need to actually run both

Always! If you're not measuring, you're not doing performance optimization. And if you think CPUs are bad: try benchmarking I/O.

Operating System (n) -- Mechanism designed specifically to prevent any meaningful performance measurement (every performance engineer ever)

If it's measurable and repeatable, it's not meaningful. If it's meaningful, it's not measurable or repeatable.

Pretty much.

Or put another way: an actual stop watch is a very meaningful performance measurement tool.

Re: Execution units are often pipelined

#59

Post by dhash »

My favorite illustrations for the concepts discussed here (in an accessible form, not the processor optimization manuals) has long been [0].

For me, this really makes working with a modern microprocessor a science, as anyone who has written benchmarks knows -- it's difficult to reason about the complex behaviour and performance cliffs without testing.

Another excellent example of the weirdness has to be JVM anatomy quarks [1]

[0] https://www.lighterra.com/papers/modernmicroprocessors/

[1] https://shipilev.net/jvm/anatomy-quarks/

Re: Execution units are often pipelined

#60

Post by gpderetta »

ajross wrote: post #52
gpderetta wrote: post #51

Earlier quoted context omitted.

> and it's a different idea than decoding multiple instructions in a cycle and sending them to one of many execution units (which is usually just called "dispatch", though "out of order execution Being able to execute multiple instructions is more properly superscalar execution, right? In-order designs are also capable of doing it and the separate execution unit do not even need to run in lockstep (consider the origi…

In-order parallel designs are "VLIW". The jargon indeed gets thick. :) But as to OO: the whole idea of issuing sequential instructions in parallel means that the hardware needs to track dependencies between them so they can't race ahead of their inputs. And if you're going to do that anyway, allowing them to retire out of order is a big performance/transistor-count win as it allows the pipeline lengths to be differen…

VLIW is again a different thing. It uses a single instruction that encodes multiple independent operations to simplify decoding and tracking, usually with exposed pipelines.

But you can have, for example, a classic in-order RISC design that allows for parallel execution. OoO renaming is not necessary for dependency tracking (in fact even scalar in order CPUs need dependency tracking to solve RAW and other hazards), it is "only" needed for executing around stalled instructions (while an in order design will stall the whole pipeline).

Again P5 (i.e the original Pentium) was a very traditional in order design, yet could execute up to two instructions per cycle.

Post reply on HN