Live data from Hacker News

Ask HN: Is big-endian dead?

news.ycombinator.com

131–140 of 193 posts

Re: Ask HN: Is big-endian dead?

#131
post #32

> Little-endian is slightly more confusing for humans... I think you meant to write “Little-endian is slightly more confusing for humans who use left-to-right languages ” as all the R-L languages also put the least significant digit on the right.

Still doesn't make sense because little endian keeps things reverse in bit groups. It doesn't make sense no matter if you read left-right, right-left, top to bottom or bottom to top.

Re: Ask HN: Is big-endian dead?

#132
Dealing with PCM audio streams (like WAV files) you sometimes find both. After all, there are thousands of codecs + media containers + multiple versions.

https://en.wikipedia.org/wiki/FFmpeg#Supported_formats

> When more than one byte is used to represent a PCM sample, the byte order (big endian vs. little endian) must be known. Due to the widespread use of little-endian Intel CPUs, little-endian PCM tends to be the most common byte orientation.

https://wiki.multimedia.cx/index.php/PCM

Re: Ask HN: Is big-endian dead?

#133

Earlier quoted context omitted.

Now that Cap'n Proto exists, why would you want to handle binary data any other way? Simply standardize on Cap'n Proto across your entire application, and problem solved :)

Baaad idea. Quassel standardized years ago on Qt’s data serialization everywhere, and it’s become a major issue now.

Story time, please.

Re: Ask HN: Is big-endian dead?

#134
post #51

As long as we use the traditional network protocols and socket API it's not "dead" dead. The other name for big-endian is "network order", after all. As way to serialize data (wire / disk format) it's becoming more common. FlatBuffers and Cap'n'Proto are the popular ones. They reduce (completely eliminate?) byte shuffling when de-serializing. In one instance I was reading a spec for an industry-specific protocol. At…

> Someone one a big endian machine, would have to do a lot more legwork to get the thing working. But given that there aren't many of those around, it was deemed an acceptable tradeoff.

Even PowerPC64 has a little-endian mode - PPC64LE.

The machines are switching over because the expensive part is turning out to be software which is already working & the costs associated with migrating it over to a different endian system.

Re: Ask HN: Is big-endian dead?

#135
> On a little-endian machine integer size casts are free -- e.g. casting a uint64_t to uint32_t just means reading the first 4 bytes of it. On big-endian machines integer size casts require pointer math.

Is it really that different? It matters on x86 if the value is in some register already, since there's no subdivision of the high part, but it shouldn't matter for memory. Most instructions support math in addressing, so whether `(char)*x` gets compiled to `[eax]` or `[eax+3]` seems not very different.

Re: Ask HN: Is big-endian dead?

#136
post #94

Earlier quoted context omitted.

That's incorrect. When you load an int64 field from a Cap'n Proto field, you are doing a 64-bit load instruction directly from the source bytes. You are not doing byte-by-byte access nor any sort of translation or "parsing". Cap'n Proto works by laying out data structures like a C compiler would, but following consistent, portable rules so that the layout is the same on all platforms. It then generates inline-able ac…

> Byte-by-byte parsing is a valid way to do parsing but not the only way. Byte-by-byte parsers tend to be slow and -- arguably, more importantly -- overly complex and rigid. It is, for example, usually very hard to do "random access" with a byte-by-byte parser, because allowing out-of-order parsing tends to blow the code complexity through the roof. I have to agree here by experiences past. If the format in question…

kentonv introduced the term 'parsing' into the discussion, not me. Originally I wasn't talking about parsing as such, just being explicit about the byte-offset, length, and ordering of any piece of data you fetch or store by doing (ptr[n] << 24) | (ptr[n+1] << 16) | (ptr[n+2] << 8) | ptr[n+3], or the corresponding write operation, if you're working with a chunk of data that came from, or is destined for, a file or the network. And if for whatever reason you want or need to work with structs, don't try to alias them onto the disk or network-bound bits. FSMs don't even come into it. It's just a matter of being a little more careful than mmap()ing into a C struct and hoping for the best.

Re: Ask HN: Is big-endian dead?

#137
post #8

Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…

Yes, LE is actually the more logical and elegant one, since bit n has weight 2^n, and byte n has a weight 256^n. The BE equivalents introduce an extra length- term into the equation (length-n).

I've always found LE vs BE to be somewhat like 0- vs 1-based array indices; one is more "conventional" to a human, but the other doesn't require a lot of contortions when expressed algebraically or algorithmically.

(With 0-based indices, element i is at address base + i * size; with 1-based indices, it's base + (i-1) * size, and it only gets worse from there as you add dimensions and other calculations.)

Re: Ask HN: Is big-endian dead?

#138
post #8

Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…

Yes, LE is actually the more logical and elegant one, since bit n has weight 2^n, and byte n has a weight 256^n. The BE equivalents introduce an extra length- term into the equation (length-n). I've always found LE vs BE to be somewhat like 0- vs 1-based array indices; one is more "conventional" to a human, but the other doesn't require a lot of contortions when expressed algebraically or algorithmically. (With 0-bas…

Or you could just decrement base.

Re: Ask HN: Is big-endian dead?

#139
I was once on a contract that involved porting satellite simulation software from SPARC/Solaris to x86/RHEL. The software was in Ada95, and it didn't take much to get it to compile on the new hardware. However, ground-to-vehicle comms (which our simulator had to handle as it needed to work with the real ground control software) took place using a packed binary format (wasting as little space as possible). Data fields frequently crossed byte boundaries (the first 10 bits are field X, the next 6 are field Y, etc.). Fun times rearranging and repacking bits.

Of course, dealing with all of the Endian issues had not been planned for and was not in the project schedule :/

Re: Ask HN: Is big-endian dead?

#140
post #51

As long as we use the traditional network protocols and socket API it's not "dead" dead. The other name for big-endian is "network order", after all. As way to serialize data (wire / disk format) it's becoming more common. FlatBuffers and Cap'n'Proto are the popular ones. They reduce (completely eliminate?) byte shuffling when de-serializing. In one instance I was reading a spec for an industry-specific protocol. At…

QUIC also uses little-endian
Post reply on HN