Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Byte Order Fallacy

commandcenter.blogspot.com

Post reply on HN
Print view View original

31–40 of 61 posts

Re: The Byte Order Fallacy

#31

Post by iscoelho »

AlotOfReading wrote: post #23
iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

A memcpy should not be slow. It should be nearly as fast as generic memory copying can be. Most of the time you shouldn't even hit the actual function, but instead a bit of code generated by the compiler that does exactly the copy you need.

memcpy is extremely slow. On any high-load Linux webserver, you can type "perf top" and see 20%~ of the CPU usage consumed by memcpy/syscalls/virtual memory.

This article is a good demonstration of the performance improvements via mmap zero-copy: https://medium.com/@kaixin667689/zero-copy-principle-and-imp...

Netflix also relies on zero-copy via kTLS & zero-copy TLS to serve 400Gbps: https://papers.freebsd.org/2021/eurobsdcon/gallatin-netflix-...

However, the performance gap can get even larger! (The kernel is historically not great at this.) For NVME & packet processors, you can see an increase of 10,000%+ in performance easily via a zero-copy implementation. See: https://www.dpdk.org https://spdk.io

Re: The Byte Order Fallacy

#32

Post by Laremere »

edflsafoiewq wrote: post #19
Laremere wrote: post #10

This is a reasonable way to do things, and I've used it before. However I just used Zig's method here, and like it a lot: https://ziglang.org/documentation/master/std/#std.io.Reader.... Given a reader (file, network, buffers can all be turned into readers), you can call readInt. It takes the type you want, and the endianess of the encoding. It's easy to write, self documents, and it's highly efficient.

If we're talking about a single int, the way you do it doesn't matter, just wrap it up in a readInt function. But if we're talking about a struct or an array, if you're byte-order aware you can do things like memcpy the whole thing around that you couldn't do by assembling it out of individual readInt calls.

Yeah it's not a hard thing to do, but I think Zig does it very cleanly.

As for reading structs, that's supported too: https://ziglang.org/documentation/master/std/#std.io.Reader....

readStructEndian will read the struct into memory, and perform the relevant byte swaps if the machine's endianness doesn't match the data format. No need to manually specify how a struct is supposed to perform the byte swap, that's all handled automatically (and efficiently) by comptime.

Re: The Byte Order Fallacy

#33

Post by neonz80 »

iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

The compiler can optimize this. See https://gcc.godbolt.org/z/hxW7hhrd7

  #include 
  uint32_t read_le_uint32(const uint8_t* p)
  {
      return p[0] | (p[1]
ends up as

  read_le_uint32(unsigned char const*):
          mov     eax, dword ptr [rdi]
          ret
This works with Clang and gcc on x86_64 (but not with MSVC).

Re: The Byte Order Fallacy

#34

Post by formerly_proven »

iscoelho wrote: post #31
AlotOfReading wrote: post #23

Earlier quoted context omitted.

A memcpy should not be slow. It should be nearly as fast as generic memory copying can be. Most of the time you shouldn't even hit the actual function, but instead a bit of code generated by the compiler that does exactly the copy you need.

memcpy is extremely slow. On any high-load Linux webserver, you can type "perf top" and see 20%~ of the CPU usage consumed by memcpy/syscalls/virtual memory. This article is a good demonstration of the performance improvements via mmap zero-copy: https://medium.com/@kaixin667689/zero-copy-principle-and-imp... Netflix also relies on zero-copy via kTLS & zero-copy TLS to serve 400Gbps: https://papers.freebsd.org/2021/e…

Any implementation of an algorithm is slow when your baseline is not performing the computation at all.

Re: The Byte Order Fallacy

#35

Post by paulddraper »

iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

> memcpy slow

Uh...

Compared to doing nothing, yes it's "slow."

Re: The Byte Order Fallacy

#36

Post by iscoelho »

neonz80 wrote: post #33
iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

The compiler can optimize this. See https://gcc.godbolt.org/z/hxW7hhrd7 #include uint32_t read_le_uint32(const uint8_t* p) { return p[0] | (p[1] ends up as read_le_uint32(unsigned char const*): mov eax, dword ptr [rdi] ret This works with Clang and gcc on x86_64 (but not with MSVC).

The purpose of zero-copy can be to avoid deserialization at all. All you do to deserialize is:

uint8_t *buf = ...; struct example_payload *payload = (struct example_payload *) buf;

That's why when you access the variables you need to byte order swap. This is absolutely not portable, I agree. I also agree that it is error-prone. However, it is the reality of a lot of performance critical software.

Re: The Byte Order Fallacy

#37

Post by iscoelho »

formerly_proven wrote: post #34
iscoelho wrote: post #31

Earlier quoted context omitted.

memcpy is extremely slow. On any high-load Linux webserver, you can type "perf top" and see 20%~ of the CPU usage consumed by memcpy/syscalls/virtual memory. This article is a good demonstration of the performance improvements via mmap zero-copy: https://medium.com/@kaixin667689/zero-copy-principle-and-imp... Netflix also relies on zero-copy via kTLS & zero-copy TLS to serve 400Gbps: https://papers.freebsd.org/2021/e…

Any implementation of an algorithm is slow when your baseline is not performing the computation at all.

Haha, it's zero-copy! I never said it was "faster-copy" (-:

Re: The Byte Order Fallacy

#38

Post by plorkyeran »

neonz80 wrote: post #33
iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

The compiler can optimize this. See https://gcc.godbolt.org/z/hxW7hhrd7 #include uint32_t read_le_uint32(const uint8_t* p) { return p[0] | (p[1] ends up as read_le_uint32(unsigned char const*): mov eax, dword ptr [rdi] ret This works with Clang and gcc on x86_64 (but not with MSVC).

Yeah, I’ve occasionally had to manually special case big/little endian code, but most of the time you can write the generic code and the optimizer will take care of it. Unless you’re doing something very complicated it’s a quite trivial optimization to perform.

Re: The Byte Order Fallacy

#39

Post by pmarreck »

iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

maybe i'm missing something because I don't code network drivers but wouldn't it be something like...

if it's little endian (on the wire), the process would be like:

    (value[0] | (value[1]
and in big endian (again, on the wire, architecture endianness irrelevant) it would be the same thing with the indices reversed, where "value" is the 4 bytes read in off the wire?

Re: The Byte Order Fallacy

#40

Post by iscoelho »

pmarreck wrote: post #39
iscoelho wrote: post #21

If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement. - mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order. - filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.) This is ab…

maybe i'm missing something because I don't code network drivers but wouldn't it be something like... if it's little endian (on the wire), the process would be like: (value[0] | (value[1] and in big endian (again, on the wire, architecture endianness irrelevant) it would be the same thing with the indices reversed, where "value" is the 4 bytes read in off the wire?

The performance would be absolutely horrendous if network drivers were programmed this way. DMA (Direct Memory Access) is all about avoiding deserialization and copies of the data.
Post reply on HN