Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Going faster than memcpy

squadrick.dev

Post reply on HN
Print view View original

41–50 of 83 posts

Re: Going faster than memcpy

#41

Post by m0th87 »

Sesse__ wrote: post #37
orlp wrote: post #32

Earlier quoted context omitted.

> Non-temporal instructions don't have anything to do with correctness. They are for cache management; a non-temporal write is a hint to the cache system that you don't expect to read this data (well, address) back soon I disagree with this statement (taken at face value, I don't necessarily agree with the wording in the OP either). Non-temporal instructions are unordered with respect to normal memory operations, so…

You mean if you access it from a different core? I believe that within the same core, you still have the normal ordering, but indeed, non-temporal writes don't have an implicit write fence after them like x86 stores normally do. In any case, if so they are potentially _less_ correct; they never help you.

There are no guarantees even if everything operates on the same core. Rust docs have some details: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._mm_sfe...

Re: Going faster than memcpy

#42

Post by m0th87 »

Sesse__ wrote: post #38
m0th87 wrote: post #33

Earlier quoted context omitted.

I work on optimizations like this at work, and yes this is largely correct. But do you have a source on this? > or (more likely) go into just some special small subsection of it reserved for non-temporal writes only. I hadn’t heard of this before. It looks like older x86 CPUs may have had a dedicated cache.

A source on what? The Intel optimization manuals explain what MOVNTQ is for. I don't think they explain in detail how it is implemented behind-the-scenes. See e.g. https://cdrdv2.intel.com/v1/dl/getContent/671200 chapter 13.5.5: “The non-temporal move instructions (MOVNTI, MOVNTQ, MOVNTDQ, MOVNTPS, and MOVNTPD) allow data to be moved from the processor’s registers directly into system memory without being also writte…

I see, thanks. I had assumed incorrectly that NT writes operated the same as NT accesses, where there is no dedicated cache.

Re: Going faster than memcpy

#44

Post by murderfs »

o11c wrote: post #20
comex wrote: post #16

Earlier quoted context omitted.

IPC libraries often specifically avoid zero-copy for security reasons. If a malicious message sender can modify the message while the receiver is in the middle of parsing it, you have to be very careful not to enable time-of-check-time-of-use attacks. (To be fair, not all use cases need to be robust against a malicious sender.)

On Linux, that's exactly what `memfd` seals are for. That said, even without seals, it's often possible to guarantee that you only read the memory once; in this case, even if the memory is technically mutating after you start, it doesn't matter since you never see any inconsistent state.

It is very easy for zero-copy IPC using sealed memfd to be massively slower than just copying, because of the cost associated with doing a TLB shootdown on munmap. In order to see a benefit over just writing into a pipe, you'd likely need to be sending gigantic blobs, mapping them in both the reader and write into an address space that isn't shared with any other threads that are doing anything, and deferring and batching munmapping (and Linux doesn't really provide you an actual way to do this, aside from mapping them all in consecutive pages with MAP_FIXED and munmapping multiple mappings with a single call).

Any realistic high-performance zero copy IPC mechanism needs to avoid changing the page tables like the plague, which means things like memfd seals aren't really useful.

Re: Going faster than memcpy

#45

Post by shakna »

kvemkon wrote: post #43

BTW, if we copy data between some device and RAM efficiently using DMA without spending CPU cycles, why we can't use DMA to copy RAM-to-RAM?

You can copy that way.

It's faster of you use the CPU, but you absolutely can just use DMA - and some embedded systems do.

Re: Going faster than memcpy

#46

Post by Sesse__ »

m0th87 wrote: post #41
Sesse__ wrote: post #37

Earlier quoted context omitted.

You mean if you access it from a different core? I believe that within the same core, you still have the normal ordering, but indeed, non-temporal writes don't have an implicit write fence after them like x86 stores normally do. In any case, if so they are potentially _less_ correct; they never help you.

There are no guarantees even if everything operates on the same core. Rust docs have some details: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._mm_sfe...

Do you have any Intel references for it? I mean, Rust has its own memory model and it will not always give the same guarantees as when writing assembler.

Re: Going faster than memcpy

#47

Post by zozbot234 »

bob1029 wrote: post #31
adwn wrote: post #17

> The operation of copying data is super easy to parallelize across multiple threads. […] This will make the copy super-fast especially if the CPU has a large core count. I seriously doubt that. Unless you have a NUMA system, a single core in a desktop CPU can easily saturate the bandwidth of the system RAM controller. If you can avoid going through main memory – e.g., when copying between the L2 caches of different…

> a single core in a desktop CPU can easily saturate the bandwidth of the system RAM controller. Modern x86 machines offer far more memory bandwidth than what a single core can consume. The entire architecture is designed on purpose to ensure this. The interesting thing to note is that this has not always been the case. The 2010s is when the transition occurred.

Some modern non-x86 machines (and maybe even some very recent x86 ones) can't even saturate their system memory bandwidth with all of their CPU cores running at full tilt, they'd need to combine both CPU and non-CPU access for absolute best performance.

Re: Going faster than memcpy

#48

Post by kvemkon »

shakna wrote: post #45
kvemkon wrote: post #43

BTW, if we copy data between some device and RAM efficiently using DMA without spending CPU cycles, why we can't use DMA to copy RAM-to-RAM?

You can copy that way. It's faster of you use the CPU, but you absolutely can just use DMA - and some embedded systems do.

> It's faster of you use the CPU

But not for AMD? E.g. 8 Zen 5 cores in the CCD have only 64 GB/s read and 32 GB/s write bandwidth, while the dual-channel memory controller in the IOD has up to 87 GB/s bandwidth.

Re: Going faster than memcpy

#49

Post by m0th87 »

Sesse__ wrote: post #46
m0th87 wrote: post #41

Earlier quoted context omitted.

There are no guarantees even if everything operates on the same core. Rust docs have some details: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._mm_sfe...

Do you have any Intel references for it? I mean, Rust has its own memory model and it will not always give the same guarantees as when writing assembler.

https://www.intel.com/content/www/us/en/docs/intrinsics-guid...

Intel's docs are unfortunately spartan, but the guarantees around program order is a hint that this is what it does.

Re: Going faster than memcpy

#50

Post by Sesse__ »

m0th87 wrote: post #49
Sesse__ wrote: post #46

Earlier quoted context omitted.

Do you have any Intel references for it? I mean, Rust has its own memory model and it will not always give the same guarantees as when writing assembler.

https://www.intel.com/content/www/us/en/docs/intrinsics-guid... Intel's docs are unfortunately spartan, but the guarantees around program order is a hint that this is what it does.

That doc is about visibility _outside the core_ (“globally visible”), so it's not what I'm looking for.

Similarly, if I look up MOVNTDQ in the Intel manuals (https://www.intel.com/content/dam/www/public/us/en/documents...), they say:

“Because the WC protocol uses a weakly-ordered memory consistency model, a fencing operation implemented with the SFENCE or MFENCE instruction should be used in conjunction with VMOVNTDQ instructions if multiple processors might use different memory types to read/write the destination memory locations”

Note _if multiple processors_.

Post reply on HN