161–170 of 193 posts
So the one that ends big (most significant byte last) should be big endian and the one that ends little should be little endian. But it's actually the opposite. It makes it difficult to remember which is which.
As long as we use the traditional network protocols and socket API it's not "dead" dead. The other name for big-endian is "network order", after all. As way to serialize data (wire / disk format) it's becoming more common. FlatBuffers and Cap'n'Proto are the popular ones. They reduce (completely eliminate?) byte shuffling when de-serializing. In one instance I was reading a spec for an industry-specific protocol. At…
In a previous job, the software I worked on had a "file format" that just consisted of dumping a raw array of structs to disk. It was obviously pretty fast, but it made cringe a lot.
What's worse, the compiler had flags to control if and how much struct padding to use, so reading a data file with a binary compiled with different padding from the one that created that file caused it to crash. Fun times...
I used to use IETF's network order (big-endian) for everything on disk or over the wire, but I switched to little-endian around 2008. If I ever have to port to a big-endian machine I'll deal with it then. I haven't regretted it yet.
Earlier quoted context omitted.
Story time, please.
I only started contributing in '14, so most of what I heard is second hand information, this is all simpkified, and some parts will likely be wrong. It's also all my own opinion, I'm not representing any project here. Basically, a decade ago a student started to work with some friends on an IRC client that integrates with a custom bouncer. Being a prototype, they just used Qt's serialization protocol between them. Ov…
Building upon an existing, well-documented, and relatively sane serialization format (protobuf, capn't proto, message pack, json, heck even bencode for all I care) is usually a good thing, and so is decoupling the messages from the details of an implementation's internals. Language and framework internal serializers (such as Python's pickle or, apparently, Qt's serializer) tend to make it harder to achieve both goals.
I find the the terms big endian and little endian confusing, feels like they should be switched around. So the one that ends big (most significant byte last) should be big endian and the one that ends little should be little endian. But it's actually the opposite. It makes it difficult to remember which is which.
Earlier quoted context omitted.
A dump of memory (several kilobytes, megabytes or whatever) is inherently big endian: it proceeds from the base address and goes up. It only looks "inherently big endian" when you print bytes on each line starting from the left. This way of printing numbers makes little sense as you end up with some kind of mixed-endian where the bytes are ordered one way and the bits another way. Start each line with bytes from the…
Everything is consistent in the big (down to the nybble and bit) endian view. The number 0x12345678 is actually the nybbles 1 2 3 4 ... which are the bits 0001 0010 0011 0100 and so on. In the hex dump 12 34 56 78 we just understand the bytes to be big-endian also at the nybble level (and bit also). That is to say the "1" can be understood to be at the lower "nybble address", relative to the "2". If that buffer is se…
00000000: 6C 6F 77 09 30 0A 66 72 65 65 09 31 32 32 0A 65 low.0.free.122.e
e.221.eerf.0.wol 65 0A 32 32 31 09 65 65 72 66 0A 30 09 77 6F 6C :00000000
[1] My first computer was a Tandy Color Computer, which had a serial port driven directly by the CPU. I learned pretty quickly which bit goes first.
Funny I just asked a question about this on SO when I discovered that CBOR - a very recent protocol - uses network byte order (big endian). A confounding decision given that no machines today use it, and it just add extra byte swapping at both ends. I can't explain it other than presumable there was some guy on the design committee who was like "but.... Unix... the 70s... network byte order... guys!" or something.
Little endian is also better for multi-word addition/subtraction. Your first operand will be the low word, which your pointer conveniently already indexes. This was a classic trick in the days of 8 and 16 bit systems, and assembly code for modern modular crypto systems still uses it. Really I think consensus is that the industry focus on big endian systems was actually a mistake. It prioritized programmer comfort in…
Yes, LE is actually the more logical and elegant one, since bit n has weight 2^n, and byte n has a weight 256^n. The BE equivalents introduce an extra length- term into the equation (length-n). I've always found LE vs BE to be somewhat like 0- vs 1-based array indices; one is more "conventional" to a human, but the other doesn't require a lot of contortions when expressed algebraically or algorithmically. (With 0-bas…
Earlier quoted context omitted.
Big-endian is the canonical form for storage and the wire because back when the Internet was designed most "pro" machines were big-endian: Sparc, old MIPS, old PPC, DEC Alpha, etc. All these are dead or dying now. It's easy enough to deal with BE files and protocols by just swapping bytes. I'm referring to hardware architectures. Are there still any big-endian chips out there? Does it still make sense to support big-…
Big-endian means that am ultra-high-speed routing fabric can receive the most significant part of an address address and possibly start making a routing decision before the other bytes of the frame have arrived. Hands down no brainer. Even the fact that headers come before payloads is a kind of "big endian", as is the fact that important information tends to occur earlier in headers. Look at a basic Ethernet frame. T…
Perhaps the biggest mistake of IPv6 was copying IPv4's header order of source followed by destination. It would have been better if the destination address came before the source address in the header.
Earlier quoted context omitted.
I only started contributing in '14, so most of what I heard is second hand information, this is all simpkified, and some parts will likely be wrong. It's also all my own opinion, I'm not representing any project here. Basically, a decade ago a student started to work with some friends on an IRC client that integrates with a custom bouncer. Being a prototype, they just used Qt's serialization protocol between them. Ov…
This is interesting, and thanks for trying to clean up the protocol, but isn't this an orthogonal issue? The problems you are describing seem to be related to using an undocumented protocol, which is unrelated to using a custom serialization format vs building upon an existing one. Building upon an existing, well-documented, and relatively sane serialization format (protobuf, capn't proto, message pack, json, heck ev…
The problem with that is that whatever format seems well-documented and relatively sane today might become an obscure, unknown protocol 10 years down the road.