Historical and obscure machines aside, there are a few things modern C++ code should take for granted, because even new systems will probably not bother breaking them: Text is encoded in UTF-8. Negative integers are twos-complement. Float is 32 bit ieee 754, double and long double are 64 bit ieee 754. Char is 8 bit, short is 16 bit, int is 32 bit, long long is 64 bit.
The Byte Order Fiasco
271–280 of 378 posts
Re: The Byte Order Fiasco
#272Earlier quoted context omitted.
Ie how do you know the target's endianness? C++20 added std::endian. Otherwise you can use a macro like this one from SDL https://github.com/libsdl-org/SDL/blob/9dc97afa7190aca5bdf92...
There have been CPU architectures where the endianness at compile time isn't necessarily sufficient. I forget which, maybe it was DEC Alpha, where the CPU could flip back and forth? I can't recall if it was a "choose at boot" or a per process change.
ARM allows dynamic changing of endianess[1].
[1]: https://developer.arm.com/documentation/dui0489/h/arm-and-th...
Re: The Byte Order Fiasco
#273Earlier quoted context omitted.
I think we really have the iPhone jailbreakers to thank for that. U.S. developers were allergic almost offended by anything that wasn't ASCII and then someone released an app that unlocked the emoji icons that Apple had originally intended only for Japan. Emoji is defined in the astral planes so almost nothing at the time was capable of understanding them, yet were so irresistible that developers worldwide who would…
I'm pretty sure Unicode was pretty widespread before the iphone/emoji popularity.
There's supporting Unicode, and 'supporting' Unicode. If you're only dealing with western languages, it's easy to fall into the trap of only 'supporting' Unicode. Proper emoji handling will put things like grapheme clusters and zero-width joiners on your map.
Re: The Byte Order Fiasco
#274Ubsan should default on. If people don't like it, then they should be made turn it off with a switch, so at least it's more likely to be run than not run. Could save a huge amount of time debugging when compilers or architecture changes. Without it, I'd say many a programmer would be caught by these subtleties in the standard. Coming from a HW background (Verilog) I'd more naturally default to masking and shifting wh…
There was a blog post and a FOSDEM presentation by (misguided) Gentoo developers a few years ago, and it was retracted, because sanitizers add their own exploitable vulnerabilities due to the way they work. https://blog.hboeck.de/archives/879-Safer-use-of-C-code-runn... https://www.openwall.com/lists/oss-security/2016/02/17/9
Sorry for my ignorance, but surely some UB being used for optimization by the compiler is compile time only. This is the part that should default on. Runtime detection is a different thing entirely, but compile time is a no brainer.
Re: The Byte Order Fiasco
#275Earlier quoted context omitted.
> One of the few reasons I ever even reached to C is the ability to slurp in data and reinterpret it as a struct, or the ability to reason in which registers things will show up and mix in some `asm` with my C. Which results in undefined behavior according to the C ISO standard. Quote: “2 All declarations that refer to the same object or function shall have compatible type; otherwise, the behavior is undefined.” From…
It should be perfectly fine to do this: union reinterpret { char raw[100]; struct myStruct interpreted; } example; read(fd, &example.raw) struct myStruct dest = interpreted; This is standard-compliant C code, and it is a common way of reading IP addresses from packets, for example.
(It should be noted that this is not valid C++ code.)
Re: The Byte Order Fiasco
#276Rust gets this right. These primitives are available for all the numeric types. u32::from_le_byte(bytes) // u32 from 4 bytes, little endian u32::from_be_byte(bytes) // u32 from 4 bytes, big endian u32::to_le_bytes(num) // u32 to 4 bytes, little endian u32::to_be_bytes(num) // u32 to 4 bytes, big endian This was very useful to me recently as I had to write the marshaling and un-marshaling for a game networking format…
Unless you are planning on running your game on a mainframe, just don’t bother with endianness for the networking. Big endian is dead for game developers. Copy entire arrays of structs onto the wire without fear! (Just #pragma pack them first)
> game on a mainframe
Maybe your program isn't a game.
Maybe you have to deal a server that uses Power, or an embedded system that uses PowerPC (or ARM or MIPS in big-endian mode).
Maybe you're running on an older architecture (SPARC, PowerPC, 68K.)
Maybe you have to deal with a pre-defined data format (e.g. TCP/IP packet headers) that uses big-endian byte ordering for some of its components.
Re: The Byte Order Fiasco
#277Earlier quoted context omitted.
Unless you are planning on running your game on a mainframe, just don’t bother with endianness for the networking. Big endian is dead for game developers. Copy entire arrays of structs onto the wire without fear! (Just #pragma pack them first)
> game on a mainframe Maybe your program isn't a game. Maybe you have to deal a server that uses Power, or an embedded system that uses PowerPC (or ARM or MIPS in big-endian mode). Maybe you're running on an older architecture (SPARC, PowerPC, 68K.) Maybe you have to deal with a pre-defined data format (e.g. TCP/IP packet headers) that uses big-endian byte ordering for some of its components.
That’s theoretically possible. But I’d be very interested in why. Especially if you are doing anything involving networking.
Re: The Byte Order Fiasco
#278It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `bswap`. Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. There is a huge mismatch between the assumptions of the C spec and actual machine code. One of the few reasons I ever even reached to C is the ability to slurp in…
Of course nobody wants C to backstab them with UB, but at the same time programmers want compilers to generate optimal code. That's the market pressure that forces optimizers to be so aggressive. If you can accept less optimized code, why aren't you using tcc? The idea of C that "just" does a straightforward machine translation breaks down almost immediately. For example, you'd want `int` to just overflow instead of…
> nobody wants C to backstab them with UB, but at the same time programmers want compilers to generate optimal code
The value of compiler optimization isn't the same thing as the value of having extensive undefined behaviour in a programming language.
Rust and Ada perform about the same as C, but lack C's many footguns.
> indexing `arr[i]` can't use 64-bit memory addressing modes
What do you mean here?
Re: The Byte Order Fiasco
#279It’s not every day you can write a blog post that calls out rob pike… ;)
Author here. I'm improving upon Rob Pike's outstanding work. Standing on the shoulders of a giant.
Totally agree. My comment was made in jest. Mad kudos to you as you clearly possess talent and humility that’s in short supply today.
Re: The Byte Order Fiasco
#280> If you program in C long enough, stuff like this becomes second nature, and it starts to almost feel inappropriate to even have macros like the above, since it might be more appropriately inlined into the specific code. Since there have simply been too many APIs introduced over the years for solving this problem. To name a few for 32-bit byte swapping alone: bswap_32, htobe32, htole32, be32toh, le32toh, ntohl, and…
Typical C culture, you would also expect that by now something like SDS would be part of the standard as well. https://github.com/antirez/sds
Adding API that introduces an entirely new string model that is incompatible with the rest of the standard library seems like a nonstarter.