It’s not every day you can write a blog post that calls out rob pike… ;)
Author here. I'm improving upon Rob Pike's outstanding work. Standing on the shoulders of a giant.
The Byte Order Fiasco
21–30 of 378 posts
Re: The Byte Order Fiasco
#22Remember how we used to have machines with a 7 bit byte? And everything was written to handle either 6, 7, or 8 bit bytes? And now we've settled on all machines being 8 bit bytes, and programmers no longer have to worry about such details? Is it time to do the same for big endian machines? Is it time to accept that all machines that matter are little endian, and the extra effort keeping everything portable to big end…
What happens is that all machines that matter are little endian but network works always in Big Endian.
We'll have to keep it as a quirk of history...
A bit like the electron has a negative charge...
Re: The Byte Order Fiasco
#23This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions.
The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C:
#include
#include
using namespace boost::endian;
unsigned char b[5] = {0x80,0x01,0x02,0x03,0x04};
int main() {
uint32_t x = *((big_uint32_t*)(b+1));
printf("%08x\n", x);
}
https://gcc.godbolt.org/z/5416oefjx
[Edit] Fun twist: the above code doesn't work where the intermediate variable x is removed because printf itself is not type safe, so no type conversion (which is when the bswap is deferred to) happens. In pure C++ when using a type safe formatting function (like fmt or iostreams) this wouldn't happen. printf will let you throw any garbage in to it. tl;dr outside embedded use cases writing C in 2021 is fucking nuts.
Re: The Byte Order Fiasco
#24Byte order is one of the great unnecessary historical fuck ups in computing. A similar one is that signedness of char is machine dependent. It's typically signed on Intel and unsigned on ARM. Sigh!
the greatest of all is lisp not being the most mainstream language, and we can only blame the lisp companies for this fiasco. in an ideal world we all would be using a lisp with parametric polymorphism. from highest level abstractions to machine level, all in one language.
Re: The Byte Order Fiasco
#25If you can assume GCC or Clang then __builtin_bswap{16,32,64} functions are provided which will be considerably more efficient, less error-prone, and easier to use than anything you can homebrew.
Given it can be done with careful code AND many processors have a single instruction to do it I’m surprised it hasn’t been added to the C standard.
Re: The Byte Order Fiasco
#26Remember how we used to have machines with a 7 bit byte? And everything was written to handle either 6, 7, or 8 bit bytes? And now we've settled on all machines being 8 bit bytes, and programmers no longer have to worry about such details? Is it time to do the same for big endian machines? Is it time to accept that all machines that matter are little endian, and the extra effort keeping everything portable to big end…
That reminds me of a project to interface with vending machines. (We built a bookshop in a vending machine that would tweet whenever it sold an item, with automated stock management.)
Vending machines have an internal protocol a little like I2C. We created a custom peripheral to bridge the machine to the web, based on a Raspberry Pi.
The protocol was defined by Coca Cola Japan in 1975 (in order to have optionality in their supply chain). It's still in use today. But because it was designed in Japan, with a need for wide characters, it assumes 9 bit bytes.
We couldn't find any way to get a Raspberry Pi to speak 9 bit bytes. The eventual solution was a custom shield that would read the bits, and reserialise to 8 bit bytes for the Pi to understand. And vice versa.
9 bit bytes. I grew up knowing that bytes had variable length, bit this was the first time I encountered it in the wild. This was 2015.
Re: The Byte Order Fiasco
#27A while back I was on a project to port a satellite simulator from SPARC/Solaris to RHEL/x64. The compressed telemetry stream that came from the satellite needed to be in big endian (and that's what the ground station software expected), and the simulator needed to mimic the behavior. This was not a problem for the old SPARC system, which naturally put everything in the correct order without any fuss, but one of the…
If memory serves correctly, ada 2012 and beyond has language level support for this. I was working on porting some code from an aviation platform to run on PC and it was all in ada 2005 so we didn't have the benefit of that available.
Re: The Byte Order Fiasco
#28If you can assume GCC or Clang then __builtin_bswap{16,32,64} functions are provided which will be considerably more efficient, less error-prone, and easier to use than anything you can homebrew.
The article explicitly shows that the provided macros are very efficient with a modern compiler. You can check on godbolt.org that they emit the same code.
Though the article only mentions bswap64 and mentioning __builtin_bswap64 would be a nice addition.
Re: The Byte Order Fiasco
#29If you can assume GCC or Clang then __builtin_bswap{16,32,64} functions are provided which will be considerably more efficient, less error-prone, and easier to use than anything you can homebrew.
But then you have to #ifdef the endianness of the target architecture. If you do it the right way as Russ Cox and Justine Tunney say, then your code can serialize and deserialize correctly regardless of the platform endianness.
Re: The Byte Order Fiasco
#30Byte order is one of the great unnecessary historical fuck ups in computing. A similar one is that signedness of char is machine dependent. It's typically signed on Intel and unsigned on ARM. Sigh!
The good thing is that Big Endian is pretty much irrelevant these days. Of all the historically Big Endian architectures, s390x is indeed the only one left that has not switched to little endian.
> The good thing is that Big Endian is pretty much irrelevant these days.
This is nonsense - many file formats are big endian.