Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance Hints

abseil.io

Post reply on HN
Print view View original

11–20 of 46 posts

Re: Performance Hints

#11

Post by VorpalWay »

xnx wrote: post #2

This formatting is more intuitive to me. L1 cache reference 2,000,000,000 ops/sec L2 cache reference 333,333,333 ops/sec Branch mispredict 200,000,000 ops/sec Mutex lock/unlock (uncontended) 66,666,667 ops/sec Main memory reference 20,000,000 ops/sec Compress 1K bytes with Snappy 1,000,000 ops/sec Read 4KB from SSD 50,000 ops/sec Round trip within same datacenter 20,000 ops/sec Read 1MB sequentially from memory 15,62…

Your suggestion confuses latency and throughput. So it isn't correct.

For example, a modern CPU will be able to execute other instructions while waiting for a cache miss, and will also be able to have multiple cache loads in flight at once (especially for caches shared between cores).

Main memory is asynchronous too, so multiple loads might be in flight, per memory channel. Same goes for all the other layers here (multiple SSD transactions in flight at once, multiple network requests, etc)

Approximately everything in modern computers is async at the hardware level, often with multiple units handling the execution of the "thing". All the way from the network and SSD to the ALUs (arithmetic logic unit) in the CPU.

Modern CPUs are pipelined (and have been since the mid to late 90s), so they will be executing one instruction, decoding the next instruction and retiring (writing out the result of) the previous instruction all at once. But real pipelines have way more than the 3 basic stages I just listed. And they can reorder, do things in parallel, etc.

Re: Performance Hints

#12

Post by jesse__ »

simonask wrote: post #4
barfoure wrote: post #3

Some of this can be reduced to a trivial form, which is to say practiced in reality on a reasonable scale, by getting your hands on a microcontroller. Not RTOS or Linux or any of that, but just a microcontroller without an OS, and learning it and learning its internal fetching architecture and getting comfortable with timings, and seeing how the latency numbers go up when you introduce external memory such as SD Card…

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

> Compilers outperform humans here almost all the time.

I'm going to be annoying and nerd-snipe you here. It's, generally, really easy to beat the compiler.

https://scallywag.software/vim/blog/simd-perlin-noise-i

Re: Performance Hints

#13

Post by jesse__ »

barfoure wrote: post #6
simonask wrote: post #4

Earlier quoted context omitted.

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

It is not about outperforming the compiler - it’s about being comfortable with measuring where your clock cycles are spent, and for that you first need to be comfortable with clock cycle scale of timing. You’re not expected to rewrite the program in assembly. But you should have a general idea given an instruction what its execution entails, and where the data is actually coming from. A read from different busses mea…

Excellent corrective summary.

"Compilers can do all these great transformations, but they can also be incredibly dumb"

-Mike Acton, CPPCON 2014

Re: Performance Hints

#14

Post by dardeaup »

simonask wrote: post #4
barfoure wrote: post #3

Some of this can be reduced to a trivial form, which is to say practiced in reality on a reasonable scale, by getting your hands on a microcontroller. Not RTOS or Linux or any of that, but just a microcontroller without an OS, and learning it and learning its internal fetching architecture and getting comfortable with timings, and seeing how the latency numbers go up when you introduce external memory such as SD Card…

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

"A lot of code can be pessimized by golfing instruction counts"

Can you explain what this phrase means?

Re: Performance Hints

#16

Post by Pannoniae »

simonask wrote: post #4
barfoure wrote: post #3

Some of this can be reduced to a trivial form, which is to say practiced in reality on a reasonable scale, by getting your hands on a microcontroller. Not RTOS or Linux or any of that, but just a microcontroller without an OS, and learning it and learning its internal fetching architecture and getting comfortable with timings, and seeing how the latency numbers go up when you introduce external memory such as SD Card…

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

Compilers massively outperform humans if the human has to write the entire program in assembly. Even if a human could write a sizable program in assembly, it would be subpar compared to what a compiler would write. This is true.

However, that doesn't mean that looking at the generated asm / even writing some is useless! Just because you can't globally outperform the compiler, doesn't mean you can't do it locally! If you know where the bottleneck is, and make those few functions great, that's a force multiplier for you and your program.

Re: Performance Hints

#17

Post by cogman10 »

jesse__ wrote: post #10
xnx wrote: post #2

This formatting is more intuitive to me. L1 cache reference 2,000,000,000 ops/sec L2 cache reference 333,333,333 ops/sec Branch mispredict 200,000,000 ops/sec Mutex lock/unlock (uncontended) 66,666,667 ops/sec Main memory reference 20,000,000 ops/sec Compress 1K bytes with Snappy 1,000,000 ops/sec Read 4KB from SSD 50,000 ops/sec Round trip within same datacenter 20,000 ops/sec Read 1MB sequentially from memory 15,62…

I prefer a different encoding: cycles/op Both ops/sec and sec/op vary on clock rate, and clock rate varies across machines, and along the execution time of your program. AFAIK, Cycles (a la _rdtsc) is as close as you can get to a stable performance measurement for an operation. You can compare it on chips with different clock rates and architectures, and derive meaningful insight. The same cannot be said for op/sec o…

Unfortunately, what you'll find if you dig into this is that cycles/op isn't as meaningful as you might imagine.

Most modern CPUs are out of order executors. That means that while a floating point operation might take 4 cycles to complete, if you put a bunch of other instructions around it like adds, divides, and multiplies, those will all finish at roughly the same time.

That makes it somewhat hard to reason about exactly how long any given set of operations will be. A FloatMul could take 4 cycles on it's own, and if you have

    FloatMul
    ADD
    MUL
    DIV
That can also take 4 cycles to finish. It's simply not as simple as saying "Let's add up the cycles for these 4 ops to get the total cycle count".

Realistically, what you'll actually be waiting on is cache and main memory. This fact is so reliable that it underpins SMT. It's why most modern CPUs will do that in some form.

Re: Performance Hints

#18

Post by CalChris »

xnx wrote: post #2

This formatting is more intuitive to me. L1 cache reference 2,000,000,000 ops/sec L2 cache reference 333,333,333 ops/sec Branch mispredict 200,000,000 ops/sec Mutex lock/unlock (uncontended) 66,666,667 ops/sec Main memory reference 20,000,000 ops/sec Compress 1K bytes with Snappy 1,000,000 ops/sec Read 4KB from SSD 50,000 ops/sec Round trip within same datacenter 20,000 ops/sec Read 1MB sequentially from memory 15,62…

I’ve seen this list many many times and I’m always surprised it doesn’t include registers.

Re: Performance Hints

#19

Post by simonask »

dardeaup wrote: post #14
simonask wrote: post #4

Earlier quoted context omitted.

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

"A lot of code can be pessimized by golfing instruction counts" Can you explain what this phrase means?

An old approach to micro-optimization is to look at the generated assembly, and trying to achieve the same thing with fewer instructions. However, modern CPUs are able to execute multiple instructions in parallel (out-of-order execution), and this mechanism relies on detecting data dependencies between instructions.

It means that the shorter sequence of instructions is not necessarily faster, and can in fact make the CPU stall unnecessarily.

The fastest sequence of instructions is the one that makes the best use of the CPU’s resources.

Re: Performance Hints

#20

Post by simonask »

Pannoniae wrote: post #16
simonask wrote: post #4

Earlier quoted context omitted.

Just be careful not to blindly apply the same techniques to a mobile or desktop class CPU or above. A lot of code can be pessimized by golfing instruction counts, hurting instruction-level parallelism and microcode optimizations by introducing false data dependencies. Compilers outperform humans here almost all the time.

Compilers massively outperform humans if the human has to write the entire program in assembly. Even if a human could write a sizable program in assembly, it would be subpar compared to what a compiler would write. This is true. However, that doesn't mean that looking at the generated asm / even writing some is useless! Just because you can't globally outperform the compiler, doesn't mean you can't do it locally ! If…

It’s absolutely not useless, I do it often as a way to diagnose various kinds of problems. But it’s extremely rare that a handwritten version actually performs better.
Post reply on HN