Ask HN: Is big-endian dead? • Hacker News Board

Re: Ask HN: Is big-endian dead?

#71

Post by kazinator » Fri, Jan 19, 2018, 9:06 PM UTC

astrodust wrote: post #66

dogecoinbase wrote: post #63

Earlier quoted context omitted.

No, cut-through mode for packet forwarding (including on layer 3 prefix matches!) is a feature in current use on L2/3 10/40/100Gb switches, primarily in HFT/HPC environments.

It's my understanding that this works with the complete address, not just the first byte, where it begins processing before the whole frame is received. On 100Gbit you're talking 0.01ns per bit, so 0.32ns for a full IPv4 address. This compared to, potentially, 15.24ns for a complete IPv4 frame.

Nevertheless; header before payload is a kind of "higher level big endian".

Re: Ask HN: Is big-endian dead?

#72

Post by IshKebab » Fri, Jan 19, 2018, 9:07 PM UTC

kazinator wrote: post #47

ajross wrote: post #8

Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…

OTOH, big-endian is better for comparison; you can tell when one integer is larger than another by looking at as little as just one bit, from the most significant end.

But comparison is done in registers which don't have a byte order. Unless you mean doing something like 256bit_a > 256bit_b which I would think it's a very rare operation. Good point though.

Re: Ask HN: Is big-endian dead?

#73

Post by amelius » Fri, Jan 19, 2018, 9:07 PM UTC

kazinator wrote: post #47

ajross wrote: post #8

Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…

OTOH, big-endian is better for comparison; you can tell when one integer is larger than another by looking at as little as just one bit, from the most significant end.

In the worst case, comparison is still like subtraction.

Re: Ask HN: Is big-endian dead?

#74

Post by jacquesm » Fri, Jan 19, 2018, 9:09 PM UTC

Gibbon1 wrote: post #68

rdtsc wrote: post #51

As long as we use the traditional network protocols and socket API it's not "dead" dead. The other name for big-endian is "network order", after all. As way to serialize data (wire / disk format) it's becoming more common. FlatBuffers and Cap'n'Proto are the popular ones. They reduce (completely eliminate?) byte shuffling when de-serializing. In one instance I was reading a spec for an industry-specific protocol. At…

> As long as we use the traditional network protocols I'm actually pissed the new LoRawan spec is big indian. What saw that my only thought was 'dicks'

Try upgrading your firmware.

Re: Ask HN: Is big-endian dead?

#75

Post by watmough » Fri, Jan 19, 2018, 9:09 PM UTC

astrodust wrote: post #57

gumby wrote: post #31

Earlier quoted context omitted.

FWIW every one of the machines you cited (“Sparc, old MIPS, old PPC, DEC Alpha, etc”) post dated the formation of the Internet (the arpanet transitioning to TCP); the Internet protocols just followed existing arpanet practice. Which was due to big-endian processors being common, but the dominant networked machine of that era was the PDP-10.

The PDP successor, the VAX, was strangely little-endian.

Could Dave Cutler have foreseen ... nahhh!

Re: Ask HN: Is big-endian dead?

#76

Post by IshKebab » Fri, Jan 19, 2018, 9:10 PM UTC

bitwize wrote: post #61

tlb wrote: post #4

Earlier quoted context omitted.

Swapping bytes is a huge pain in the butt! When reading large binary files, it's so convenient and efficient to be able to mmap and make struct pointers right into the file. You can even deliver those files to web browsers and use JS's TypedArray to get random access into them. (That requires a bit more than simply little-endian. It requires struct alignment and floating point formats to be the same. But with only a…

> Swapping bytes is a huge pain in the butt! When reading large binary files, it's so convenient and efficient to be able to mmap and make struct pointers right into the file. No no no don't do this don't do this don't do this. This is how horrors and abominations like .doc, .xls, .psd happen. The correct way to handle binary data is to unpack it into the struct byte by byte. The reason for this is that when you defi…

No, dumping internal representations to disk is orthogonal to zero-static data formats. You can have a well defined format that doesn't require byte-by-byte parsing.

Look up FlatBuffers and CapnProto.

Re: Ask HN: Is big-endian dead?

#77

Post by MichaelMoser123 » Fri, Jan 19, 2018, 9:10 PM UTC

deepnotderp wrote: post #21

I think a couple of networking chips still use it.

cavium is big endian mips.

Re: Ask HN: Is big-endian dead?

#78

Post by IshKebab » Fri, Jan 19, 2018, 9:11 PM UTC

kazinator wrote: post #71

astrodust wrote: post #66

Earlier quoted context omitted.

It's my understanding that this works with the complete address, not just the first byte, where it begins processing before the whole frame is received. On 100Gbit you're talking 0.01ns per bit, so 0.32ns for a full IPv4 address. This compared to, potentially, 15.24ns for a complete IPv4 frame.

Nevertheless; header before payload is a kind of "higher level big endian".

Err, no.

Re: Ask HN: Is big-endian dead?

#79

Post by ajross » Fri, Jan 19, 2018, 9:13 PM UTC

kazinator wrote: post #47

ajross wrote: post #8

Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…

OTOH, big-endian is better for comparison; you can tell when one integer is larger than another by looking at as little as just one bit, from the most significant end.

That's only an scalar code optimization though. You can still do comparison in constant time on little endian data, you just lack the ability to early-exit from the loop. In the modern world of SIMD data, that optimization often doesn't exist. Fair enough point though.

Re: Ask HN: Is big-endian dead?

#80

Post by IshKebab » Fri, Jan 19, 2018, 9:14 PM UTC

gumby wrote: post #32

> Little-endian is slightly more confusing for humans... I think you meant to write “Little-endian is slightly more confusing for humans who use left-to-right languages ” as all the R-L languages also put the least significant digit on the right.

Honestly I find little endian less confusing. The only confusing thing is how little endian value are written in hex because within each byte they are written as big endian.

Other than that little endian makes more sense. The fact that Arabic numbers are written in big endian is weird. Nobody questions it because it's all anyone is taught.