Live data from Hacker News

The Byte Order Fiasco

justine.lol

171–180 of 378 posts

Re: The Byte Order Fiasco

#171

It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `bswap`. Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. There is a huge mismatch between the assumptions of the C spec and actual machine code. One of the few reasons I ever even reached to C is the ability to slurp in…

You don’t have to mask and shift. You can memcpy and then byte swap in a function. It will get inlined as mov/bswap.

Practically speaking, common compilers have intrinsics for bswap. The memcpy function can be thought of as an intrinsic for unaligned load/store.

Re: The Byte Order Fiasco

#172
post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

As a very minor counterpoint: I like C because frankly it’s fun. I wouldn’t start a web browser or maybe even an operating system in it today, but as a language for messing around I find it rewarding. I also think it is incredibly instructive in a lot of ways. I am not a C++ developer but ANSI C has a special place in my heart.

Also, I will say that when it comes to programming Arduinos and ESP8266/ESP32 chips, I still find that C is my go to despite things like Alia, MicroPython, etc. I think it’s possible that once Zig supports those devices fully that I might move over. But in the meantime I guess I’ll keep minding my off by one errors.

Re: The Byte Order Fiasco

#173
In her first sentence, the phrase “the C / C++ programming language” is no longer correct: C++20 requires two’s complement signed integers.

C++ 20 is quite new so I would assume that very few people know this yet.

C and C++ obviously differ a lot, but by that phrase she clearly means “the part where then two languages overlap”. The C++ committee has been willing to break C compatibility in a few ways (not every valid C program is a valid C++ program), and this has been true for a while.

Re: The Byte Order Fiasco

#174
post #110

Earlier quoted context omitted.

https://fgiesen.wordpress.com/2014/10/25/little-endian-vs-bi...

Why would one choose the memory representation of the number based on the advantages of the internal ALU wiring? Of all those reasons, the only one I can make sense of is the "I can’t transparently widen fields after the fact!", and that one is way too niche to explain anything.

I don’t understand? Why not make the memory representation sympathetic with the operations you’re going to do on it? It’s the raison d’être of computers to compute and to do it fast.

Another example: memory representation of pixels in GPUs which are swizzled to make computations efficient

Re: The Byte Order Fiasco

#175
post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

I find you missed the point of the post and the issues described in it.

In my estimation, libraries like boost are way too big and way too clever and they create more problems than they solve. Also, they don't make me happy.

You're overfocusing on a "problem" that is almost completely irrelevant for most of programming. Big endian is rare to be found (almost no hardware to be found, but some file formats and networking APIs have big-endian data in them). Where you still meet it, you don't do endianness conversions willy-nilly. You have only a few lines in a huge project that should be concerned with it. Similar situation for dealing with aligned reads.

So, with boost you end up with a huge slow-compiling dependency to solve a problem using obscure implicit mechanisms that almost no-one understands or can even spot (I would never have guessed that your line above seems to handle misalignment or byte swapping).

This approach is typical for a large group of C++ programmers, who seem to like to optimize for short code snippets, cleverness, and/or pedantry.

The actual issue described in the post was the UB that is easy to hit when doing bit shifting, caused by the implicit conversions that are defined in C. While this is definitely an unhappy situation, it's easy enough to avoid this using plain C syntax (cast expression to unsigned before shifting), using not more code than the boost-type cast in your above code.

The fact that the UB is so easy to hit doesn't call for excessive abstraction, but simply a revisit of some of the UB defined in C, and how compiler writers exploit it.

(Anecdata: I've written a fair share of C code, while not compression or encryption algorithms, and personally I'm not sure I've ever hit one of the evil cases of UB. I've hit Segmentation faults or had Out-of-bounds accesses, sure, but personally I've never seen the language or compilers "haunt me".)

Re: The Byte Order Fiasco

#176

Earlier quoted context omitted.

I feel like big endian is more _intuitive_ because that's what our number notation has evolved to be. But more _natural_ is little endian because, well, it's just more straightforward to have the digits' magnitude be in ascending order (2^0, 2^1, 2^2, 2^3...) instead of putting it in reverse. Plus you encounter less roadblocks in practice with little endian (e.g. address changes with casts) which is often a sign of g…

I'm curious how you're defining "natural", and if you think ISO-8601 is the reverse of "natural" too. All human number systems I've ever seen write numbers out as big Endian (yes, even Roman numerals), so I'm really struggling to see how that wouldn't be considered natural.

It seems like it would be a more natural for representing the number when communicating with a human.

But that's not what we're doing here, so it's not entirely relevant.

Re: The Byte Order Fiasco

#177
post #173

In her first sentence, the phrase “the C / C++ programming language” is no longer correct: C++20 requires two’s complement signed integers. C++ 20 is quite new so I would assume that very few people know this yet. C and C++ obviously differ a lot, but by that phrase she clearly means “the part where then two languages overlap”. The C++ committee has been willing to break C compatibility in a few ways (not every valid…

It hasn't been true since C99, at least -- C++ didn't adopt C99 designated initializers.

Re: The Byte Order Fiasco

#178
post #173

In her first sentence, the phrase “the C / C++ programming language” is no longer correct: C++20 requires two’s complement signed integers. C++ 20 is quite new so I would assume that very few people know this yet. C and C++ obviously differ a lot, but by that phrase she clearly means “the part where then two languages overlap”. The C++ committee has been willing to break C compatibility in a few ways (not every valid…

What chips can be targeted by C compilers today that don't use 2's complement?

Re: The Byte Order Fiasco

#179
post #63
post #38

Earlier quoted context omitted.

Assuming the macros or your giant expression are correct. But you might as well use the compiler intrinsics which you know are both correct and the most efficient possible, and get on with your life.

Sorry I'd rather place my faith in arithmetic rather than someone's API provided the compiler is smart enough to understand the arithmetic and optimize accordingly.

"Someone" here is the same compiler you're trusting to optimize your giant arithmetic expression of the same idea. Your statement is internally inconsistent.

Re: The Byte Order Fiasco

#180

Isn't the 'modern' solution to memcpy into a temp and swap the bytes in that? C++ has added/will add std::launder and std::bless to deal with this issue

> Isn't the 'modern' solution to memcpy into a temp and swap the bytes in that?

Or just use the endian.h / sys/endian.h routines, which do the right thing (be32dec / be32enc / whatever). memcpy+swap is fine, and easier to get right than the author's giant expressions, but you might as well use the named routines that do exactly what you want already.

Post reply on HN