Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Going faster than memcpy

squadrick.dev

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Going faster than memcpy

#61

Post by toast0 »

kvemkon wrote: post #43

BTW, if we copy data between some device and RAM efficiently using DMA without spending CPU cycles, why we can't use DMA to copy RAM-to-RAM?

DMA works for devices, because the device does the memory access. RAM to RAM DMA would need something to do the accesses.

The other reason DMA works for devices is because it is asynchronous. You give a device a command and some memory to do it with, it does the thing and lets you know. Most devices can't complete commands instantaneously, so we know we have to queue things and then go do something else. Often when doing memcpy, we want to use the copied memory immediately... if it were a DMA, you'd need to submit the request and wait for it to complete before you continued... If your general purpose DMA engine is a typical device, you're probably doing a syscall to the kernel, which would submit the command (possibly through a queue), suspend your process, schedule something else and there may be delay before getting scheduled again when the DMA is complete.

If async memcpy was what was wanted, it could make sense, but that feels pretty hard to use.

Re: Going faster than memcpy

#62

Post by CyberDildonics »

EGreg wrote: post #57

Wait, I thought memcpy would have launched some sort of built-in mechanism (parallelized or whatever) to copy in RAM. Just indicate the start and length. Why would the CPU need to keep issuing copy instructions?

I thought memcpy would have launched some sort of built-in mechanism

Where did you get this impression?

Re: Going faster than memcpy

#63

Post by EGreg »

CyberDildonics wrote: post #62
EGreg wrote: post #57

Wait, I thought memcpy would have launched some sort of built-in mechanism (parallelized or whatever) to copy in RAM. Just indicate the start and length. Why would the CPU need to keep issuing copy instructions?

I thought memcpy would have launched some sort of built-in mechanism Where did you get this impression?

From my college days, which were quite long ago. And working with Win32 "BitBlt" requests to the OS, etc.

And also, it would just make sense. If copying entire blocks or memory pages, such as "BitBlt", is one command, why would I need CPU cycles to actually do it? It would seem like the lowest hanging fruit to automate in SDRAM

It just seems like the easiest example of SIMD

Re: Going faster than memcpy

#64

Post by commandlinefan »

I've gotten a lot of gains in this area in the past by just - not memcpy'ing. A good percentage of the time, somebody assumes that they need to copy something somewhere when in fact, the original never gets referenced. I can often get away with reading a buffer off the wire, inserting null terminators to turn bits of the buffer into proper C-style strings and just using them in-place.

Re: Going faster than memcpy

#65

Post by t00 »

commandlinefan wrote: post #64

I've gotten a lot of gains in this area in the past by just - not memcpy'ing. A good percentage of the time, somebody assumes that they need to copy something somewhere when in fact, the original never gets referenced. I can often get away with reading a buffer off the wire, inserting null terminators to turn bits of the buffer into proper C-style strings and just using them in-place.

That is a really good advice, copying data everywhere makes only sense if the data will be mutated. I only wonder why, why C-style strings were invented with 0 termination instead of varint prefix, this would have saved so much copying and so many bugs knowing the string length upfront.

Re: Going faster than memcpy

#66

Post by AlotOfReading »

t00 wrote: post #65
commandlinefan wrote: post #64

I've gotten a lot of gains in this area in the past by just - not memcpy'ing. A good percentage of the time, somebody assumes that they need to copy something somewhere when in fact, the original never gets referenced. I can often get away with reading a buffer off the wire, inserting null terminators to turn bits of the buffer into proper C-style strings and just using them in-place.

That is a really good advice, copying data everywhere makes only sense if the data will be mutated. I only wonder why, why C-style strings were invented with 0 termination instead of varint prefix, this would have saved so much copying and so many bugs knowing the string length upfront.

Pascal strings have the issue that you need to agree on an int size to cross an ABI boundary, unless you want to limit all strings to 255 characters and what the prefix means is ambiguous if you have variable length characters (e.g. Unicode). These were severe enough that Pascal derivatives all added null terminated strings.

Took a bit for languages to develop the distinction between string length in characters and bytes that allows us to make it work today. In that time C derivatives took over the world.

Re: Going faster than memcpy

#67

Post by zozbot234 »

toast0 wrote: post #61
kvemkon wrote: post #43

BTW, if we copy data between some device and RAM efficiently using DMA without spending CPU cycles, why we can't use DMA to copy RAM-to-RAM?

DMA works for devices, because the device does the memory access. RAM to RAM DMA would need something to do the accesses. The other reason DMA works for devices is because it is asynchronous. You give a device a command and some memory to do it with, it does the thing and lets you know. Most devices can't complete commands instantaneously, so we know we have to queue things and then go do something else. Often when d…

> DMA works for devices, because the device does the memory access. RAM to RAM DMA would need something to do the accesses.

Isn't a blitter exactly that sort of device? Assuming that it can access the relevant RAM, why couldn't that be used for general-purpose memory copying operations?

Re: Going faster than memcpy

#68

Post by CyberDildonics »

EGreg wrote: post #63
CyberDildonics wrote: post #62

Earlier quoted context omitted.

I thought memcpy would have launched some sort of built-in mechanism Where did you get this impression?

From my college days, which were quite long ago. And working with Win32 "BitBlt" requests to the OS, etc. And also, it would just make sense. If copying entire blocks or memory pages, such as "BitBlt", is one command, why would I need CPU cycles to actually do it? It would seem like the lowest hanging fruit to automate in SDRAM It just seems like the easiest example of SIMD

These are contradictory things. SIMD instructions are still regular instructions, not some concurrent system for copying. When you say command, maybe you meant a windows OS function that was similar to memcpy. An OS function and individual CPU instructions are two different thing. There is something called DMA, but I don't know how much that is used for memory to memory copies.

Re: Going faster than memcpy

#69

Post by toast0 »

zozbot234 wrote: post #67
toast0 wrote: post #61

Earlier quoted context omitted.

DMA works for devices, because the device does the memory access. RAM to RAM DMA would need something to do the accesses. The other reason DMA works for devices is because it is asynchronous. You give a device a command and some memory to do it with, it does the thing and lets you know. Most devices can't complete commands instantaneously, so we know we have to queue things and then go do something else. Often when d…

> DMA works for devices, because the device does the memory access. RAM to RAM DMA would need something to do the accesses. Isn't a blitter exactly that sort of device? Assuming that it can access the relevant RAM, why couldn't that be used for general-purpose memory copying operations?

Yes, but PCs have only rarely had general purpose blitters. They were integrated in some video cards, but that's more or less like DMA; Intel had one for a while recently [1]; FreeBSD loads a driver for it on my Xeon L5640 hosted server, but I don't see any evidence that anything actually uses it. and I'm not sure there was enough actual performance improvement enabled by offloading copies, so Intel stopped including these. Linux marked their driver as broken because it caused issues with copy-on-write [2]

[1] https://lwn.net/Articles/162966/ [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Re: Going faster than memcpy

#70

Post by commandlinefan »

t00 wrote: post #65
commandlinefan wrote: post #64

I've gotten a lot of gains in this area in the past by just - not memcpy'ing. A good percentage of the time, somebody assumes that they need to copy something somewhere when in fact, the original never gets referenced. I can often get away with reading a buffer off the wire, inserting null terminators to turn bits of the buffer into proper C-style strings and just using them in-place.

That is a really good advice, copying data everywhere makes only sense if the data will be mutated. I only wonder why, why C-style strings were invented with 0 termination instead of varint prefix, this would have saved so much copying and so many bugs knowing the string length upfront.

That reminds me of one of my favorite vulnerabilities. A security researcher named Moxie Marlinspike managed to register an SSL cert for .com by submitting a certificate request for the domain .com\0mygooddomain.com. The CA looked at the (length prefixed) ASN.1 subject name and saw that it had a legitimate domain, they accepted it, but most implementations treated the subject name as a C-delimited string and stopped parsing at the null terminator.
Post reply on HN