Ha, I love the project name "Shadesmar". Journey before destination, friend. :crossed-wrists:
Going faster than memcpy
51–60 of 83 posts
Re: Going faster than memcpy
#52If I understand that chart at the end it looks like the better performance is only for small buffer sizes which fit in the cache (4k) but if you are looking at big buffers the stdlib copy performs about the same as the optimized copy that he writes.
Re: Going faster than memcpy
#53The graph at the end seems pretty dubious. For example, for the AvxUnrollCopier, why does data transfer speed jump to >120gb/s for 4kb, then down to ~50gb/s for 32kb, then down to <20gb/s for 16mb? It just doesn't make sense.
Re: Going faster than memcpy
#54Earlier quoted context omitted.
You can copy that way. It's faster of you use the CPU, but you absolutely can just use DMA - and some embedded systems do.
> It's faster of you use the CPU But not for AMD? E.g. 8 Zen 5 cores in the CCD have only 64 GB/s read and 32 GB/s write bandwidth, while the dual-channel memory controller in the IOD has up to 87 GB/s bandwidth.
The issue is that a DMA setup:
A: requires the DMA system to know about each user process memory mappings (ie hardware support understanding CPU pagetables)
B: spend time going from user-kernelmode and back (we invented the entire io_uring and other mechanisms to avoid that).
To some extent I guess the IOMMU's available to modern graphics cards solve it partially but I'm not sure that it's a free lunch (ie it might be partially in driver/OS level to manage mappings for this).
Re: Going faster than memcpy
#55Earlier quoted context omitted.
You mean if you access it from a different core? I believe that within the same core, you still have the normal ordering, but indeed, non-temporal writes don't have an implicit write fence after them like x86 stores normally do. In any case, if so they are potentially _less_ correct; they never help you.
There are no guarantees even if everything operates on the same core. Rust docs have some details: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._mm_sfe...
[deleted]
Re: Going faster than memcpy
#56The graph at the end seems pretty dubious. For example, for the AvxUnrollCopier, why does data transfer speed jump to >120gb/s for 4kb, then down to ~50gb/s for 32kb, then down to <20gb/s for 16mb? It just doesn't make sense.
The L1 cache is faster than the L3 cache. Does it need to be anything more complicated than that?
Re: Going faster than memcpy
#57Wait, I thought memcpy would have launched some sort of built-in mechanism (parallelized or whatever) to copy in RAM.
Just indicate the start and length. Why would the CPU need to keep issuing copy instructions?
Re: Going faster than memcpy
#58Wait, I thought memcpy would have launched some sort of built-in mechanism (parallelized or whatever) to copy in RAM. Just indicate the start and length. Why would the CPU need to keep issuing copy instructions?
The poster has a Zen 2, where this is only optimal for large copies. For newer Intel, glibc might indeed choose to use REP MOVSB more often.
Re: Going faster than memcpy
#59Conclusion Stick to `std::memcpy`. It delivers great performance while also adapting to the hardware architecture, and makes no assumptions about the memory alignment. ---- So that's five minutes I'll never get back. I'd make an exception for RISC-V machines with "RVV" vectors, where vectorised `memcpy` hasn't yet made it into the standard library and a simple ... 0000000000000000 : 0: 86aa mv a3,a0 0000000000000002…
You could read the article and end up disagreeing with it. The value is in grokking over the details and not whether the insight changes your decisions. It can just make your decisions more grounded in data
Re: Going faster than memcpy
#60Wait, I thought memcpy would have launched some sort of built-in mechanism (parallelized or whatever) to copy in RAM. Just indicate the start and length. Why would the CPU need to keep issuing copy instructions?
The problem is that the built-in mechanism is often microcode, which is still slower than plain machine code in some cases.
There are some interesting writings from a former architect of the Pentium Pro on the reasons for this. One is apparently that the microcode engine often lacked branch prediction, so handling special cases in the microcode was slower than compare/branch in direct code. REP MOVS has a bunch of such cases due to the need to handle overlapping copies, interrupts, and determining when it should switch to cache line sized non-temporal accesses.
More recent Intel CPUs have enhanced REP MOVS support with faster microcode and a flag indicating that memcpy() should rely on it more often. But people have still found cases where if the relative alignment between source and destination is just right, a manual copy loop is still noticeably faster than REP MOVS.