Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimising Memset and Memcpy

twitter.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Optimising Memset and Memcpy

#21

Post by WalterBright »

astrobe_ wrote: post #9
easytiger wrote: post #4

Earlier quoted context omitted.

Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning

CPU makers add so many complicated features to let users optimize their programs that it takes too much knowledge for mere mortals to optimize their programs - That you have to rely on heuristics and measures, as it is usually and wisely advised, is a bit unsatisfying. That's sort of ironic, in a way. I used to do assembly and count cycles, but now I wouldn't dare; it's hardcore compiler and library makers stuff. It'…

A good heuristic is simply minimize the number of instructions.

Re: Optimising Memset and Memcpy

#22

Post by mhh__ »

WalterBright wrote: post #21
astrobe_ wrote: post #9

Earlier quoted context omitted.

CPU makers add so many complicated features to let users optimize their programs that it takes too much knowledge for mere mortals to optimize their programs - That you have to rely on heuristics and measures, as it is usually and wisely advised, is a bit unsatisfying. That's sort of ironic, in a way. I used to do assembly and count cycles, but now I wouldn't dare; it's hardcore compiler and library makers stuff. It'…

A good heuristic is simply minimize the number of instructions.

I doubt it'd be an issue for memcpy but minimizing the size of a loop body can lead to very counterintuitive speedups now processors have LSDs (loop stream detectors).

Re: Optimising Memset and Memcpy

#24

Post by boulos »

Hi Nadav!

Seems like a lot of the benefit also comes from "Intel processors since no longer care about aligned vs unaligned loads and stores" (so older code that spends effort lining this up doesn't benefit in the small cases up to 512 bytes).

Does that rationale still hold for larger blocks and/or hitting the cache lines awkwardly? Like make something where you'll have plenty of 64-byte copies which are offset enough that they are off alignment and pollute the neighboring cache line (probably most visible for memset).

Re: Optimising Memset and Memcpy

#25

Post by azinman2 »

Living higher up in the stack, there’s a bit of this I don’t really understand, but I’d like to. Can someone explain what’s going on here with a bit more information/explanation of these techniques?

Re: Optimising Memset and Memcpy

#26

Post by dan-robertson »

I was a little confused by the assembler in the fourth tweet. Here is what I think is going on. First the code:

  1. vxorps %xmm0, %xmm0, %xmm0
  2. vmovups %ymm0, 11(%rdi)
  3. vmovups %ymm0, (%rdi)
  4. vzeroupper
So line 1 sets xmm0, a 128 bit/16 byte register, to 0 (by xoring it with itself). I suppose the convention is for the caller to save the register if it wants it. The thing that confused me is that the way 256 bit/32 byte registers work is that ymmX represents the 256 bit register of which xmmX is the lower half. Similar to the way %eax is the lower 32 bits of %rax. I suppose the upper half of %ymm0 is 0 by ABI.

So line 2 then writes out this 32-byte register (of zeros) starting at buffer[11] and ending at buffer[42]. The actual instruction name stands for “vector move unaligned packed single precision floats” and I don’t really understand why the precision or float type matters.

Line 3 writes it from buffer[0] to buffer[31]. I guess the overlap isn’t expensive.

Line 4 zeros the upper half of the register which is a no-op in this case but might be useful as it breaks the dependency chain so the cpu can know nothing else will need to use the current upper-half values (but isn’t it useful to break the chain on the lower half too?).

Re: Optimising Memset and Memcpy

#27

Post by jeffbee »

mhh__ wrote: post #22
WalterBright wrote: post #21

Earlier quoted context omitted.

A good heuristic is simply minimize the number of instructions.

I doubt it'd be an issue for memcpy but minimizing the size of a loop body can lead to very counterintuitive speedups now processors have LSDs (loop stream detectors).

If you make them too small it can lead to counterintuitive slowdowns.

Re: Optimising Memset and Memcpy

#28

Post by kierank »

astrobe_ wrote: post #9
easytiger wrote: post #4

Earlier quoted context omitted.

Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning

CPU makers add so many complicated features to let users optimize their programs that it takes too much knowledge for mere mortals to optimize their programs - That you have to rely on heuristics and measures, as it is usually and wisely advised, is a bit unsatisfying. That's sort of ironic, in a way. I used to do assembly and count cycles, but now I wouldn't dare; it's hardcore compiler and library makers stuff. It'…

This is definitely not correct. For many SIMDable workloads it's possible to achieve order of magnitude improvements over the compiler.

Re: Optimising Memset and Memcpy

#29

Post by ckennelly »

floatboth wrote: post #10
bonzini wrote: post #6

On most new processors, Intel just suggests using REP STOSB and REP MOVSB for memset and memcpy respectively. That's not in the benchmark as far as I can see.

To define "new" more precisely, there is an "Enhanced REP MOVSB" flag in cpuid that tells you you can use those. It is set from Ivy Bridge and Zen 3 up. It's still not always faster: https://stackoverflow.com/questions/43343231/enhanced-rep-mo...

As mentioned in that Stack Overflow post, though, things change again with FSRM (Fast Short Rep Mov).

While there are still startup costs, the overhead of calling a function (especially via a PLT) and incurring instruction cache misses is hard to demonstrate in a microbenchmark, while rep movsb encodes more compactly than many flavors of call. In an actual application though, the "slower" but smaller implementation can often win (https://research.google/pubs/pub50338.pdf and https://research.google/pubs/pub48320.pdf)

Re: Optimising Memset and Memcpy

#30

Post by pjmlp »

cout wrote: post #11
astrobe_ wrote: post #9

Earlier quoted context omitted.

CPU makers add so many complicated features to let users optimize their programs that it takes too much knowledge for mere mortals to optimize their programs - That you have to rely on heuristics and measures, as it is usually and wisely advised, is a bit unsatisfying. That's sort of ironic, in a way. I used to do assembly and count cycles, but now I wouldn't dare; it's hardcore compiler and library makers stuff. It'…

I have hand-written asm for use in production code, and I would do it again. I also knew exactly what cpu my code would be running on. If I were publishing the code to run on a variety of hardware, I would be very cautious. Writing your own crypto is very different; the stakes are higher if you get it wrong.

Including the microcode version currently installed into the CPU and gone through the release notes, alongside a profile like V-Tune?
Post reply on HN