Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Going faster than memcpy

squadrick.dev

Post reply on HN
Print view View original

21–30 of 83 posts

Re: Going faster than memcpy

#21

Post by hugh-avherald »

adwn wrote: post #17

> The operation of copying data is super easy to parallelize across multiple threads. […] This will make the copy super-fast especially if the CPU has a large core count. I seriously doubt that. Unless you have a NUMA system, a single core in a desktop CPU can easily saturate the bandwidth of the system RAM controller. If you can avoid going through main memory – e.g., when copying between the L2 caches of different…

I've experienced modest but significant improvements in speed using very basic pragma omp section style parallelizing of this sort of thing.

Re: Going faster than memcpy

#22

Post by Sesse__ »

There's an error here: “NT instructions are used when there is an overlap between destination and source since destination may be in cache when source is loaded.”

Non-temporal instructions don't have anything to do with correctness. They are for cache management; a non-temporal write is a hint to the cache system that you don't expect to read this data (well, address) back soon, so it shouldn't push out other things in the cache. They may skip the cache entirely, or (more likely) go into just some special small subsection of it reserved for non-temporal writes only.

Re: Going faster than memcpy

#23

Post by duped »

comex wrote: post #16
waschl wrote: post #2

Thought about zero-copy IPC recently. In order to avoid memcopy for the complete chain, I guess it would be best if the sender allocates its payload directly on the shared memory when it’s created. Is this a standard thing in such optimized IPC and which libraries offer this?

IPC libraries often specifically avoid zero-copy for security reasons. If a malicious message sender can modify the message while the receiver is in the middle of parsing it, you have to be very careful not to enable time-of-check-time-of-use attacks. (To be fair, not all use cases need to be robust against a malicious sender.)

What's the threat model where a malicious message sender has write access to shared memory

Re: Going faster than memcpy

#24

Post by adwn »

hugh-avherald wrote: post #21
adwn wrote: post #17

> The operation of copying data is super easy to parallelize across multiple threads. […] This will make the copy super-fast especially if the CPU has a large core count. I seriously doubt that. Unless you have a NUMA system, a single core in a desktop CPU can easily saturate the bandwidth of the system RAM controller. If you can avoid going through main memory – e.g., when copying between the L2 caches of different…

I've experienced modest but significant improvements in speed using very basic pragma omp section style parallelizing of this sort of thing.

Do you remember any specifics? For example, the size of the copy, whether it was a NUMA system, or the total bandwidth of your system RAM?

Re: Going faster than memcpy

#25

Post by hmry »

duped wrote: post #23
comex wrote: post #16

Earlier quoted context omitted.

IPC libraries often specifically avoid zero-copy for security reasons. If a malicious message sender can modify the message while the receiver is in the middle of parsing it, you have to be very careful not to enable time-of-check-time-of-use attacks. (To be fair, not all use cases need to be robust against a malicious sender.)

What's the threat model where a malicious message sender has write access to shared memory

How would someone send a message over shared memory without write access to that memory?

Re: Going faster than memcpy

#27

Post by a_t48 »

waschl wrote: post #2

Thought about zero-copy IPC recently. In order to avoid memcopy for the complete chain, I guess it would be best if the sender allocates its payload directly on the shared memory when it’s created. Is this a standard thing in such optimized IPC and which libraries offer this?

I've looked into this a bit - the big blocker isn't on the transport/IPC library, but the serializer itself, assuming you _also_ want to support serializing messages to disk or over network. It's a bit of a pickle - at least in C++, tying an allocator to a structure and its children is an ugly mess. And what happens if you do something like resize a string? Does it mean a whole new allocation? I've (partially) solved it before for single process IPC by having a concept of a sharable structure and its serialization type, you could do the same for shared memory. One could also use a serializer that offers promises around allocations, FlatBuffer might fit the bill. There's also https://github.com/Verdant-Robotics/cbuf but I'm not sure how well maintained it is right now, publicly.

As for allocation - it looks like Zenoh might offer the allocation pattern necessary. https://zenoh-cpp.readthedocs.io/en/1.0.0.5/shm.html TBH most of the big wins come from not copying big blocks of memory around from sensor data and the like. A thin header and reference to a block of shared memory containing an image or point cloud coming in over UDS is likely more than performant enough for most use cases. Again, big wins from not having to serialize/deserialize the sensor data.

Another pattern which I haven't really seen anywhere is handling multiple transports - at one point I had the concept of setting up one transport as an allocator (to put into shared memory or the like) - serialize once to shared memory, hand that serialized buffer to your network transport(s) or your disk writer. It's not quite zero copy but in practice most zero copy is actually at least one copy on each end.

(Sorry, this post is a little scatterbrained, hopefully some of my points come across)

Re: Going faster than memcpy

#28

Post by IshKebab »

hmry wrote: post #25
duped wrote: post #23

Earlier quoted context omitted.

What's the threat model where a malicious message sender has write access to shared memory

How would someone send a message over shared memory without write access to that memory?

I think he meant what's the scenario where you're using IPC via shared memory and don't trust both processes. Basically it only applies if the processes are running as two different users. (I think Android does that a lot?)

Re: Going faster than memcpy

#29

Post by kragen »

o11c wrote: post #20
comex wrote: post #16

Earlier quoted context omitted.

IPC libraries often specifically avoid zero-copy for security reasons. If a malicious message sender can modify the message while the receiver is in the middle of parsing it, you have to be very careful not to enable time-of-check-time-of-use attacks. (To be fair, not all use cases need to be robust against a malicious sender.)

On Linux, that's exactly what `memfd` seals are for. That said, even without seals, it's often possible to guarantee that you only read the memory once; in this case, even if the memory is technically mutating after you start, it doesn't matter since you never see any inconsistent state.

Thanks for the reference! I had been wondering if there was a way to do this on Linux for years. https://lwn.net/Articles/591108/ seems to be the relevant note?

Re: Going faster than memcpy

#30

Post by kragen »

duped wrote: post #23
comex wrote: post #16

Earlier quoted context omitted.

IPC libraries often specifically avoid zero-copy for security reasons. If a malicious message sender can modify the message while the receiver is in the middle of parsing it, you have to be very careful not to enable time-of-check-time-of-use attacks. (To be fair, not all use cases need to be robust against a malicious sender.)

What's the threat model where a malicious message sender has write access to shared memory

When you are using the shared memory to communicate with an untrusted sender. Examples might include:

- browser main processes that don't trust renderer processes

- window system compositors that don't trust all windowed applications, and vice versa

- database servers that don't trust database clients, and vice versa

- message queue brokers that don't trust publishers and subscribers, and vice versa

- userspace filesystems that don't trust normal user processes

Post reply on HN