Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Byte Order Fiasco

justine.lol

Post reply on HN
Print view View original

131–140 of 378 posts

Re: The Byte Order Fiasco

#131

Post by vlmutolo »

> If you program in C long enough, stuff like this becomes second nature, and it starts to almost feel inappropriate to even have macros like the above, since it might be more appropriately inlined into the specific code. Since there have simply been too many APIs introduced over the years for solving this problem. To name a few for 32-bit byte swapping alone: bswap_32, htobe32, htole32, be32toh, le32toh, ntohl, and htonl which all have pretty much the same meaning.

> Now you don't need to use those APIs because you know the secret.

This sentiment seems problematic. The solution shouldn't be "we just have to educate the masses of C programmers on how to properly deal with endianness". That will never happen.

The solution should be "It's in the standard library. Go look there and don't think too hard." C is sufficiently low-level, and endianness problems sufficiently common, that I would expect that kind of routine to be available.

Re: The Byte Order Fiasco

#132

Post by owl57 »

BlueTemplar wrote: post #95
wongarsu wrote: post #56

Earlier quoted context omitted.

People were holding off on transitioning because pointers use twice as much space in x64. If bytes had quadrupled in space with x64 we would still be using 32 bit software everywhere

Well, obviously it would have delayed the transition. However you can only go so far with 4Go-limited memory. And do you have examples of still widely used 8-bit sized data formats ?

I assume you wrote this comment in UTF-8 over HTTP (ASCII-based) and TLS (lots of uint8 fields).

Re: The Byte Order Fiasco

#133

Post by pjmlp »

jart wrote: post #94
nly wrote: post #87

Earlier quoted context omitted.

For the record, she's not really saying people shouldn't learn this low level stuff... just that 'intro to C++' shouldn't be teaching this stuff first The biggest problem with C++ in industry is that people tend to write "C/C++" when it deserves to be recognized as a language in its own right.

One does not simply introduce C++. It's the most insanely hardcore language there is. I wouldn't have stood any chance understanding it had it not been for my gentle introduction with C for several years.

Really?

Apparently the first year students at my university didn't had any issue going from Standard Pascal to C++, in the mid-90's.

Proper C++ was taught using our string, vector and collection classes, given that we were still a couple of years away from ISO C++ being fully defined.

C style programming with low level tricks were only introduced later as advanced topics.

Apparently thousands of students managed to get going the remaining 5 years of the degree.

Re: The Byte Order Fiasco

#134

Post by MauranKilom »

tails4e wrote: post #6

Ubsan should default on. If people don't like it, then they should be made turn it off with a switch, so at least it's more likely to be run than not run. Could save a huge amount of time debugging when compilers or architecture changes. Without it, I'd say many a programmer would be caught by these subtleties in the standard. Coming from a HW background (Verilog) I'd more naturally default to masking and shifting wh…

> Ubsan should default on

> Could save a huge amount of time debugging when compilers or architecture changes.

I'm assuming we come from very different backgrounds, but it's not clear to me how switching compilers or architectures is so common that hardening code against it by default is appropriate. I would think that switching compilers or architectures is generally done very deliberately, so instrumenting code with UBsan for that transition would be the right thing to do?

Re: The Byte Order Fiasco

#135

Post by DonHopkins »

londons_explore wrote: post #15

Remember how we used to have machines with a 7 bit byte? And everything was written to handle either 6, 7, or 8 bit bytes? And now we've settled on all machines being 8 bit bytes, and programmers no longer have to worry about such details? Is it time to do the same for big endian machines? Is it time to accept that all machines that matter are little endian, and the extra effort keeping everything portable to big end…

We used to have machines with arbitrarily sized bytes, and 36 bit words!

http://pdp10.nocrew.org/docs/instruction-set/Byte.html

>In the PDP-10 a "byte" is some number of contiguous bits within one word. A byte pointer is a quantity (which occupies a whole word) which describes the location of a byte. There are three parts to the description of a byte: the word (i.e., address) in which the byte occurs, the position of the byte within the word, and the length of the byte.

>A byte pointer has the following format:

     000000 000011 1 1 1111 112222222222333333
     012345 678901 2 3 4567 890123456789012345
     _________________________________________
    |      |      | | |    |                  |
    | POS  | SIZE |U|I| X  |        Y         |
    |______|______|_|_|____|__________________|
>POS is the byte position: the number of bits from the right end of the byte to the right end of the word. SIZE is the byte size in bits.

>The U field is ignored by the byte instructions.

>The I, X and Y fields are used, just as in an instruction, to compute an effective address which specifies the location of the word containing the byte.

"If you're not playing with 36 bits, you're not playing with a full DEC!" -DIGEX (Doug Humphrey)

http://otc.umd.edu/staff/humphrey

Re: The Byte Order Fiasco

#136

Post by pjmlp »

vlmutolo wrote: post #131

> If you program in C long enough, stuff like this becomes second nature, and it starts to almost feel inappropriate to even have macros like the above, since it might be more appropriately inlined into the specific code. Since there have simply been too many APIs introduced over the years for solving this problem. To name a few for 32-bit byte swapping alone: bswap_32, htobe32, htole32, be32toh, le32toh, ntohl, and…

Typical C culture, you would also expect that by now something like SDS would be part of the standard as well.

https://github.com/antirez/sds

Re: The Byte Order Fiasco

#137

Post by themulticaster »

foldr wrote: post #101
nly wrote: post #23

This is why, in 2021, the mantra that C is a good language for these low level byte twiddling tasks needs to die. Dealing with alignment and endianness properly requires a language that allows you to build abstractions. The following is perfectly well defined in C++, despite looking like almost the same as the original unsafe C: #include #include using namespace boost::endian; unsigned char b[5] = {0x80,0x01,0x02,0x0…

What are the advantages of this over a simple function with the following signature? uint32_t read_big_uint32(char *bytes); Having a big_uint32_t type seems wrong to me conceptually. You should either deal with sequences of bytes with a defined endianness or with native 32-bit integers of indeterminate endianness (assuming that your code is intended to be endian neutral). Having some kind of halfway house just confus…

I agree, but a little nitpick: A sequence of bytes does not have a defined endianness. Only groups of more than one bytes (i.e. half words, words, double words or whatever you want to call them) have an endianness.

In practice, most projects (e.g. the Linux kernel or the socket interface) differentiate between host (indeterminate) byte order and a specific byte order (e.g. network byte order/big endian).

Re: The Byte Order Fiasco

#138

Post by ta988 »

jart wrote: post #94
nly wrote: post #87

Earlier quoted context omitted.

For the record, she's not really saying people shouldn't learn this low level stuff... just that 'intro to C++' shouldn't be teaching this stuff first The biggest problem with C++ in industry is that people tend to write "C/C++" when it deserves to be recognized as a language in its own right.

One does not simply introduce C++. It's the most insanely hardcore language there is. I wouldn't have stood any chance understanding it had it not been for my gentle introduction with C for several years.

Yes this is the curse of knowledge, people that know c++ by their exposure to it for decades are usually unable to bring any new comer to it.

Re: The Byte Order Fiasco

#139

Post by stefan_ »

raverbashing wrote: post #32
genmon wrote: post #26

Earlier quoted context omitted.

That reminds me of a project to interface with vending machines. (We built a bookshop in a vending machine that would tweet whenever it sold an item, with automated stock management.) Vending machines have an internal protocol a little like I2C. We created a custom peripheral to bridge the machine to the web, based on a Raspberry Pi. The protocol was defined by Coca Cola Japan in 1975 (in order to have optionality in…

Well you could bit bang and the 9 bits wouldn't be an issue. (Even if you had a tiny PIC microcontroler just to do that) This is best solvable the closer to the device in question and in the simplest way possible.

The irony is that while a tiny PIC can do bit banging easily, the mighty Pi will struggle with it.

Re: The Byte Order Fiasco

#140

Post by kangalioo »

ben509 wrote: post #124
bombcar wrote: post #72

Earlier quoted context omitted.

Isn’t big endian a bit more natural considered on a bit level? The bits start from highest to lowest on a serial connection.

Big-endian is natural when you're comparing numbers, which is probably why people represent numbers in a big-endian fashion. Little-endian is natural with casts because the address doesn't change, and it's the order in which addition takes place.

I feel like big endian is more _intuitive_ because that's what our number notation has evolved to be.

But more _natural_ is little endian because, well, it's just more straightforward to have the digits' magnitude be in ascending order (2^0, 2^1, 2^2, 2^3...) instead of putting it in reverse.

Plus you encounter less roadblocks in practice with little endian (e.g. address changes with casts) which is often a sign of good natural design

Post reply on HN