Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Byte Order Fiasco

justine.lol

Post reply on HN
Print view View original

41–50 of 378 posts

Re: The Byte Order Fiasco

#41

Post by captainmuon »

It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `bswap`. Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. There is a huge mismatch between the assumptions of the C spec and actual machine code.

One of the few reasons I ever even reached to C is the ability to slurp in data and reinterpret it as a struct, or the ability to reason in which registers things will show up and mix in some `asm` with my C.

I think there should really be a dialect of C(++) where the machine model is exactly the physical machine. That doesn't mean the compiler can't do optimizations, but it shouldn't do things like prove code as UB and fold everything to a no-op. (Like when you defensively compare a pointer to NULL that according to spec must not be NULL, but practically could be...)

`-fno-strict-overflow -fno-strict-aliasing -fno-delete-null-pointer-checks` gets you halfway there, but it would really only be viable if you had a blessed `-std=high-level-assembler` or `-std=friendly-c` flag.

Re: The Byte Order Fiasco

#42

Post by bonzini »

js8 wrote: post #39
bonzini wrote: post #31

Earlier quoted context omitted.

Little endian has the advantage that you can read the low bits of data without having to adjust the address. So you can for example do long addition in memory order rather than having to go backwards, or (with an appropriate representation such as ULEB128) in one pass without knowing the size.

Maybe I am biased working on mainframes, but I would personally take big endian over little endian. The reason is when reading a hex dump, I can easily read the binary integers from left to right.

That's the only thing that BE has over LE.

But for example bitmaps in BE are a huge source of bugs, as readers and writers need to agree on the size to use for memory operations.

"SIMD in a word" (e.g. doing strlen or strcmp with 32- or 64-bit memory accesses) might have mostly fallen out of fashion these days, but it's also more efficient in LE.

Re: The Byte Order Fiasco

#43

Post by kstenerud »

bregma wrote: post #7
amelius wrote: post #3

Byte order is one of the great unnecessary historical fuck ups in computing. A similar one is that signedness of char is machine dependent. It's typically signed on Intel and unsigned on ARM. Sigh!

And which is the correct byte ordering, pray tell?

Big endian is easier for humans to read when looking at a memory dump, but little endian has many useful features in binary encoding schemes due to the low byte being first.

I used to like big endian more, but after deep investigation I now prefer little endian for any encoding schemes.

Re: The Byte Order Fiasco

#44

Post by vladharbuz »

nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

Correct me if I'm wrong, but your example is just using a library to do the same task, rather than illustrating any difference between C and C++. If you want to pull boost in to do this, that's great, but that hardly seems like a fair comparison to the OP, since instead of implementing code to solve this problem yourself you're just importing someone else's code.

Re: The Byte Order Fiasco

#45

Post by nly »

captainmuon wrote: post #41

It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `bswap`. Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. There is a huge mismatch between the assumptions of the C spec and actual machine code. One of the few reasons I ever even reached to C is the ability to slurp in…

> I think there should really be a dialect of C(++) where the machine model is exactly the physical machine.

Sounds great, until you have to rewrite all your software to go from x86-64 to ARM

Re: The Byte Order Fiasco

#46

Post by jart »

nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

C is perfect for these problems. I like teaching the endian serialization problem because it broaches so many of the topics that are key to understanding C/C++ in general. Even if we choose to spend the majority of our time plumbing together functions written by better men, it's nice to understand how the language is defined so we could write those functions, even if we don't need to.

Re: The Byte Order Fiasco

#48

Post by ChrisSD »

BlueTemplar wrote: post #36
genmon wrote: post #26

Earlier quoted context omitted.

That reminds me of a project to interface with vending machines. (We built a bookshop in a vending machine that would tweet whenever it sold an item, with automated stock management.) Vending machines have an internal protocol a little like I2C. We created a custom peripheral to bridge the machine to the web, based on a Raspberry Pi. The protocol was defined by Coca Cola Japan in 1975 (in order to have optionality in…

We really should have moved to 32 bit bytes when moving to 64 bit words. Would have simplified Unicode considerably.

Not really. Unicode is a variable width abstract encoding; a single character can be made up of multiple code points.

For Unicode, 32-bit bytes would be an incredibly wasteful in memory encoding.

Re: The Byte Order Fiasco

#49

Post by hvdijk »

dataflow wrote: post #40
pabs3 wrote: post #18

I wonder if those macros work with middle-endian systems.

Is this a joke or am I just unaware of any systems out there that are "middle-endian"..?!

There are no current middle-endian systems but they used to exist. The PDP-11 is the most famous one. The macros would work on all systems, but as only very old systems are middle-endian, they also have old compilers so may not be able to optimise it as well.

Re: The Byte Order Fiasco

#50

Post by nly »

rwmj wrote: post #8

If you can assume GCC or Clang then __builtin_bswap{16,32,64} functions are provided which will be considerably more efficient, less error-prone, and easier to use than anything you can homebrew.

My favourite builtins are the overflow checked integer operations:

https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins...

Post reply on HN