Optimising Memset and Memcpy
twitter.com
Optimising Memset and Memcpy
1–10 of 30 posts
Re: Optimising Memset and Memcpy
#2Link to Github for easier reading
https://github.com/nadavrot/memset_benchmark
Re: Optimising Memset and Memcpy
#3A whole page of benchmark results without even listing which x86 he was testing??? This varies a lot from one generation to the next and between Intel & AMD.
There are a ton of tradeoffs that have to be made in the microarchitecture and in general the focus is real code and normal expected sizes rather than benchmarks. Big page multiples are common and then variable sized calls that can be totally unaligned and random sizes.
A proper study of this would test a pile of different mixes of sizes and alignments and test on a bunch of different processor generations.
And then you get results where a hand coded memcpy loop is faster in benchmarks, but in practice in a big program you might find that 'rep movs' is better because of the smaller code footprint. When you only optimize the few cases that matter.
The general rule is that the compiler and system libraries do a really good job optimizing for processors that shipped 10 years ago. So architects get to make decisions about how to help these operations. You can add a nifty feature that makes it faster (like CLZERO) but while it is good for benchmarks it will takes years after you ship before normal programs benefit. You might optimized 'rep movs' but then you find that since it was slow in the past most real system librarys don't call it.
Re: Optimising Memset and Memcpy
#4A whole page of benchmark results without even listing which x86 he was testing??? This varies a lot from one generation to the next and between Intel & AMD. There are a ton of tradeoffs that have to be made in the microarchitecture and in general the focus is real code and normal expected sizes rather than benchmarks. Big page multiples are common and then variable sized calls that can be totally unaligned and rando…
Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics.
We are very much entering the age where mechanical sympathy is waning
Re: Optimising Memset and Memcpy
#5A whole page of benchmark results without even listing which x86 he was testing??? This varies a lot from one generation to the next and between Intel & AMD. There are a ton of tradeoffs that have to be made in the microarchitecture and in general the focus is real code and normal expected sizes rather than benchmarks. Big page multiples are common and then variable sized calls that can be totally unaligned and rando…
Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning
Also the benchmark do not seem to use the same set of sizes for all benchmarks (nor use a fixed random seed) so repeatability and comparability seem questionable.
I guess if you run it often enough it could still give useful numbers, but I understand the author is picking the best run.
Also I don't see to be any attempt at avoiding compiler optimisations.
Benchmarking is hard
Re: Optimising Memset and Memcpy
#6On most new processors, Intel just suggests using REP STOSB and REP MOVSB for memset and memcpy respectively. That's not in the benchmark as far as I can see.
Re: Optimising Memset and Memcpy
#7A whole page of benchmark results without even listing which x86 he was testing??? This varies a lot from one generation to the next and between Intel & AMD. There are a ton of tradeoffs that have to be made in the microarchitecture and in general the focus is real code and normal expected sizes rather than benchmarks. Big page multiples are common and then variable sized calls that can be totally unaligned and rando…
Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning
I guess that you are referring to other projects because the benchmarks in this repo use a stable seed, turbo disabled, physical machine, both random and stable sizes, etc.
Re: Optimising Memset and Memcpy
#8Earlier quoted context omitted.
Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning
Also the benchmark do not seem to use the same set of sizes for all benchmarks (nor use a fixed random seed) so repeatability and comparability seem questionable. I guess if you run it often enough it could still give useful numbers, but I understand the author is picking the best run. Also I don't see to be any attempt at avoiding compiler optimisations. Benchmarking is hard
Grep the code for rand_reset. The code uses a fixed seed, fixed sizes for all programs, stable nop baseline, etc. also the pointer indirection blocks compiler optimizations.
Re: Optimising Memset and Memcpy
#9A whole page of benchmark results without even listing which x86 he was testing??? This varies a lot from one generation to the next and between Intel & AMD. There are a ton of tradeoffs that have to be made in the microarchitecture and in general the focus is real code and normal expected sizes rather than benchmarks. Big page multiples are common and then variable sized calls that can be totally unaligned and rando…
Half the benchmarks i see posted here are done on AWS instances or the authors laptop with power stepping/cstates, unisolated cores etc etc likely all enabled too, let alone basic rigour around hardware specifics. We are very much entering the age where mechanical sympathy is waning
CPU makers add so many complicated features to let users optimize their programs that it takes too much knowledge for mere mortals to optimize their programs - That you have to rely on heuristics and measures, as it is usually and wisely advised, is a bit unsatisfying. That's sort of ironic, in a way.
I used to do assembly and count cycles, but now I wouldn't dare; it's hardcore compiler and library makers stuff. It's like "don't do your own crypto (optimization)".
Everyone knows why it is so, though - we cannot solve the problem by throwing more Gigahertz at it.
Re: Optimising Memset and Memcpy
#10On most new processors, Intel just suggests using REP STOSB and REP MOVSB for memset and memcpy respectively. That's not in the benchmark as far as I can see.
To define "new" more precisely, there is an "Enhanced REP MOVSB" flag in cpuid that tells you you can use those. It is set from Ivy Bridge and Zen 3 up.
It's still not always faster: https://stackoverflow.com/questions/43343231/enhanced-rep-mo...