Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is big-endian dead?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 193 posts

Re: Ask HN: Is big-endian dead?

#51

Post by rdtsc »

As long as we use the traditional network protocols and socket API it's not "dead" dead. The other name for big-endian is "network order", after all.

As way to serialize data (wire / disk format) it's becoming more common. FlatBuffers and Cap'n'Proto are the popular ones. They reduce (completely eliminate?) byte shuffling when de-serializing.

In one instance I was reading a spec for an industry-specific protocol. At first I was looking at it and thinking "What the... they are padding stuff in a strange way, and using little-endian for the data". Then it suddenly dawned on me, that they've designed the spec to be whatever GCC on x86-64 Linux machine would do to layout C the structures in memory.

So someone very lazy could just define a struct and cast into it as data comes from the wire. Someone one a big endian machine, would have to do a lot more legwork to get the thing working. But given that there aren't many of those around, it was deemed an acceptable tradeoff.

Re: Ask HN: Is big-endian dead?

#53

Post by simias »

BrandoElFollito wrote: post #43
samat wrote: post #37

Earlier quoted context omitted.

Not true. In arabic languages you write text R-L, but numbers are L-R.

Do you mean that you read a text from the right to the left, and when you meet a number you jump to is leftmost digit, then read it LR, then jump agzin to its left (where there is a space) and resume reading text?

If you think about it in left-to-right languages reading numbers requires backtracking because you have to see how many digits there are to know if you're reading units, thousands, millions, billions...

Take 12,345,567 for instance, if you want to read it aloud you first have to go all the way to the end to figure out that there are three groups of numbers, so the first one is million, then thousands, then units. If you were only given a truncated version of the number it's impossible to read it aloud. So in a way the arabic represents number actually makes some sense and it could be argue that we write them backwards.

Furthermore IIRC from a discussion with an arabic friend (from the middle east, not sure if it's different in NA) the most formal way of reading numbers aloud is actually backwards (or little endian I guess) compared to the way we do in english. So if I understood correctly for 1,234 you'd say "four and thirty and two hundred and a thousand" or something similar. Don't quote me on that though because I know almost nothing about the arabic language and this is from a discussion I've had many years ago.

Re: Ask HN: Is big-endian dead?

#54

Post by [deleted] »

drewg123 wrote: post #12
api wrote: post #3

Earlier quoted context omitted.

Big-endian is the canonical form for storage and the wire because back when the Internet was designed most "pro" machines were big-endian: Sparc, old MIPS, old PPC, DEC Alpha, etc. All these are dead or dying now. It's easy enough to deal with BE files and protocols by just swapping bytes. I'm referring to hardware architectures. Are there still any big-endian chips out there? Does it still make sense to support big-…

Small pedantic correction from a former Alpha kernel hacker: DEC Alpha was Big Endian only for some rare Cray systems. Everything else was Little Endian.

[deleted]

Re: Ask HN: Is big-endian dead?

#55

Post by arnsholt »

kazinator wrote: post #46

The "casting" advantage basically just hides bugs. Because it's often not okay to access just 16 or 8 bits of a 32 bit number; the value is being truncated and that could be a bug. However, when the number's range is small enough, such code appears to work. Finding the bug is delayed until larger values occur, which is not until its deployed in the field or someone thinks of writing better test cases. Anyway, if it i…

I ran into a bug in that area a while back. Our development environment has a binding to Winsock32.dll for socket stuff, and as it turns out the bindings were written so long ago that many of the return values are specced to be 16-bit ints. As a result, our application would work fine almost all of the time, but fail mysteriously with "socket operation on non-socket" after about 8000 sockets have been opened. As I understand it 8000 is because file descriptors are a kind of bump-the-pointer scheme internally and pointers are 8 bytes wide on 64-bit platform.

Re: Ask HN: Is big-endian dead?

#57

Post by astrodust »

gumby wrote: post #31
api wrote: post #3

Earlier quoted context omitted.

Big-endian is the canonical form for storage and the wire because back when the Internet was designed most "pro" machines were big-endian: Sparc, old MIPS, old PPC, DEC Alpha, etc. All these are dead or dying now. It's easy enough to deal with BE files and protocols by just swapping bytes. I'm referring to hardware architectures. Are there still any big-endian chips out there? Does it still make sense to support big-…

FWIW every one of the machines you cited (“Sparc, old MIPS, old PPC, DEC Alpha, etc”) post dated the formation of the Internet (the arpanet transitioning to TCP); the Internet protocols just followed existing arpanet practice. Which was due to big-endian processors being common, but the dominant networked machine of that era was the PDP-10.

The PDP successor, the VAX, was strangely little-endian.

Re: Ask HN: Is big-endian dead?

#58

Post by astrodust »

kazinator wrote: post #48
api wrote: post #3

Earlier quoted context omitted.

Big-endian is the canonical form for storage and the wire because back when the Internet was designed most "pro" machines were big-endian: Sparc, old MIPS, old PPC, DEC Alpha, etc. All these are dead or dying now. It's easy enough to deal with BE files and protocols by just swapping bytes. I'm referring to hardware architectures. Are there still any big-endian chips out there? Does it still make sense to support big-…

Big-endian means that am ultra-high-speed routing fabric can receive the most significant part of an address address and possibly start making a routing decision before the other bytes of the frame have arrived. Hands down no brainer. Even the fact that headers come before payloads is a kind of "big endian", as is the fact that important information tends to occur earlier in headers. Look at a basic Ethernet frame. T…

Maybe that mattered on serial connections, but on a 10Gbit switch you're talking nonsense.

Re: Ask HN: Is big-endian dead?

#60

Post by macintux »

drewg123 wrote: post #12
api wrote: post #3

Earlier quoted context omitted.

Big-endian is the canonical form for storage and the wire because back when the Internet was designed most "pro" machines were big-endian: Sparc, old MIPS, old PPC, DEC Alpha, etc. All these are dead or dying now. It's easy enough to deal with BE files and protocols by just swapping bytes. I'm referring to hardware architectures. Are there still any big-endian chips out there? Does it still make sense to support big-…

Small pedantic correction from a former Alpha kernel hacker: DEC Alpha was Big Endian only for some rare Cray systems. Everything else was Little Endian.

Many, many years ago I worked for Progeny, and we contracted with HP to help bring Debian to Itanium.

We quickly discovered that a fair bit of Linux software (or at least Debian packaging of same) only worked properly on 64-bit DEC Alpha because the architecture was Little Endian; 32-bit pointers would typically "just work(™)" because the code was grabbing the first 32 bits, and the other half of the pointer addresses was typically 0x0.

Since we were porting to Big Endian hardware (just now discovered Itanium was selectable, not sure why it was Big Endian in our case) we were getting a fair bit of null pointer exceptions.

Anyway, ancient history, but that was my only direct exposure to endian-madness, so it has stuck with me.

Post reply on HN