Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Byte Order Fiasco

justine.lol

Post reply on HN
Print view View original

191–200 of 378 posts

Re: The Byte Order Fiasco

#191

Post by professoretc »

jeffreygoesto wrote: post #189
nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

Wouldn't that cast be UB because it is type punning?

char* is a allowed to alias to other pointer types.

Re: The Byte Order Fiasco

#192

Post by [deleted] »

cbmuser wrote: post #75
captainmuon wrote: post #41

It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `bswap`. Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. There is a huge mismatch between the assumptions of the C spec and actual machine code. One of the few reasons I ever even reached to C is the ability to slurp in…

> One of the few reasons I ever even reached to C is the ability to slurp in data and reinterpret it as a struct, or the ability to reason in which registers things will show up and mix in some `asm` with my C. Which results in undefined behavior according to the C ISO standard. Quote: “2 All declarations that refer to the same object or function shall have compatible type; otherwise, the behavior is undefined.” From…

[deleted]

Re: The Byte Order Fiasco

#193

Post by einpoklum »

londons_explore wrote: post #15

Remember how we used to have machines with a 7 bit byte? And everything was written to handle either 6, 7, or 8 bit bytes? And now we've settled on all machines being 8 bit bytes, and programmers no longer have to worry about such details? Is it time to do the same for big endian machines? Is it time to accept that all machines that matter are little endian, and the extra effort keeping everything portable to big end…

> Is it time to accept that all machines that matter are little endian.

Well, no, because it's not the case. SPARC is big-endian, and a bunch of IBM processors. ARM processors are mostly bi-endian.

> Is it time to do the same for big endian machines?

No. Not just because of their prevalence, but because there isn't a compelling reason why everything should be little- endian.

Re: The Byte Order Fiasco

#194

Post by raphlinus »

jstimpfle wrote: post #175
nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

I find you missed the point of the post and the issues described in it. In my estimation, libraries like boost are way too big and way too clever and they create more problems than they solve. Also, they don't make me happy. You're overfocusing on a "problem" that is almost completely irrelevant for most of programming. Big endian is rare to be found (almost no hardware to be found, but some file formats and networki…

I agree with the bulk of this post.

Re the anecdata at the end. Have you ever run your code through the sanitizers? I have. CVE-2016-2414 is one of my battle scars, and I consider myself a pretty good programmer who is aware of security implications.

Re: The Byte Order Fiasco

#195

Post by kstenerud »

tom_mellior wrote: post #117
kstenerud wrote: post #76

Earlier quoted context omitted.

The only benefit to big endian is that it's easier for humans to read in a hex dump. Little endian on the other hand has many tricks available to it for building encoding schemes that are efficient on the decoder side.

Could you elaborate on these tricks? This sounds interesting. The only thing I'm aware of that's neat in little endian is that if you want the low byte (or word or whatever suffix) of a number stored at address a, then you can simply read a byte from exactly that address. Even if you don't know the size of the original number.

I've posted in some other replies, but a few:

- Long addition is possible across very large integers by just adding the bytes and keeping track of the carry.

- Encoding variable sized integers is possible through an easy algorithm: set aside space in the encoded data for the size, then encode the low bits of the value, shift, repeat until value = 0. When done, store the number of bytes you wrote to the earlier length field. The length calculation comes for free.

- Decoding unaligned bits into big integers is easy because you just store the leftover bits in the next value of the bigint array and keep going. With big endian, you're going high bits to low bits, so once you pass to more than one element in the bigint array, you have to start shifting across multiple elements for every piece you decode from then on.

- Storing bit-encoded length fields into structs becomes trivial since it's always in the low bit, and you can just incrementally build the value low-to-high using the previously decoded length field. Super easy and quick decoding, without having to prepare specific sized destinations.

Re: The Byte Order Fiasco

#197

Post by occamrazor »

zabzonk wrote: post #168
joppy wrote: post #147

Earlier quoted context omitted.

Why is it an issue any more than say, order of fields in a struct is an issue? In one case you read bytes off the disk by doing ((b[0] << 8) | b[1]) (or equivalent), with the order reversed the other way around. Any application-level (say, not a compiler, debugger, etc) program should not even need to know the native byte order, it should only need to know the encoding that the file it’s trying to read used.

> order of fields in a struct This is defined in C to be the order the fields are declared in.

But the padding rules between fields are a mess.

Re: The Byte Order Fiasco

#198

Post by occamrazor »

BenoitEssiambre wrote: post #53
tssva wrote: post #13

Earlier quoted context omitted.

Network byte order is big endian so it is far from being pretty much irrelevant these days.

Also, this might be irrelevant at the cpu level, but within a byte, bits are usually displayed most significant bit first, so with little endian you end up with bit order: 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 instead of 15 to 0 This is because little endian is not how humans write numbers. For consistency with little endianness we would have to switch to writing "one hundred and twenty three" as 321

In German _ein hundert drei und zwanzig_, literally _one hundred three and twenty_. The hardest part is are telephone numbers, that are usually given in blocks of two digits.

Re: The Byte Order Fiasco

#199

Post by 0x000000E2 »

nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

By the same token, I think most uses for C++ these days are nuts. If you're doing a greenfield project 90% of the time it's better to use Rust.

C++ has a multitude of its own pitfalls. Some of the C programmer hate for C++ is justified. After all, it's just C with a pre-processing stage in the end.

There's good reasons why many C projects never considered C++ but are already integrating the nascent Rust. I always hated low level programming until Rust made it just as easy and productive as high level stuff

Re: The Byte Order Fiasco

#200

Post by marcosdumay »

enqk wrote: post #174
marcosdumay wrote: post #153

Earlier quoted context omitted.

Why would one choose the memory representation of the number based on the advantages of the internal ALU wiring? Of all those reasons, the only one I can make sense of is the "I can’t transparently widen fields after the fact!", and that one is way too niche to explain anything.

I don’t understand? Why not make the memory representation sympathetic with the operations you’re going to do on it? It’s the raison d’être of computers to compute and to do it fast. Another example: memory representation of pixels in GPUs which are swizzled to make computations efficient

> I don’t understand? Why not make the memory representation sympathetic with the operations you’re going to do on it?

There's no reason to, as there's no reason not to. It's basically irrelevant.

If carrier passing is so important, why can't you just mirror your transistors and operate on the same wires, but on the opposite order? Well, you can, and it's trivial. (And, by the way, carrier passing isn't important. High performances ALU pass carrier only though blocks, that can appear anywhere. And the wiring of those isn't even planar, so how you arrange them isn't a showstopper.)

Post reply on HN