Earlier quoted context omitted.
> The other name for big-endian is "network order", after all. I wonder where this started... I always suspected it was Sun with their "The Network is the Computer" motto, and of course they would define "network byte order" as what they used. Today, it drives me crazy that we're constantly swapping before sending across the network and then swapping again when it's received.
This may be the best ensemble of answers you'll get to that question: https://retrocomputing.stackexchange.com/questions/2652/when... The internet was big-endian from the start, and that was probably because DEC machines were big-endian.
Ask HN: Is big-endian dead?
151–160 of 193 posts
Re: Ask HN: Is big-endian dead?
#152Earlier quoted context omitted.
Baaad idea. Quassel standardized years ago on Qt’s data serialization everywhere, and it’s become a major issue now.
Story time, please.
Basically, a decade ago a student started to work with some friends on an IRC client that integrates with a custom bouncer. Being a prototype, they just used Qt's serialization protocol between them. Over the years the project grew, at some point nokia funded development, it became Kubuntu's default IRC client (but only in the client+bouncer in one binary version, so without all the advantages), nokia was bought and closed and the department sold to BMW, and at some point.
Now, people tried writing third-party clients for this. And this became a minor issue, because the protocol was never documented. In favt, Qt's serialization was used for storing configs on disk, some blobs in the database, and over the network. It was later wrapped in TLS and deflate, and even at some point array-of-struct was turned into struct-of-array for the pattern during initialisation.
Either way, someone tried writing a mobile client for it, decided the protocol was insane, and instead built his own, almost identical client/bouncer system with an Erlang backend and json as protocol. This grew, and became IRCCloud.
Now, other people again tried developing third party clients for quassel. An android client was developed, but development was messy, and over the years, it stalled, because they reverse engineered the protocol, semi-successfully, and at some point didn't have enough time left, and gave up.
People reversed the protocol partially over the years again for pyquassel and quasselc/quasselbots/quassel-irssi.
Around then, another person tried reversing the protocol, and reimplementing it in JS for a webclient, which after a while became quassel-webserver.
Back then there was a lot of talk about replacing the protocol, but it was never done yet.
I, a user of irccloud around then, was annoyed with the costs (still being in high school myself, I couldn't afford the 4$/month, and the free tier wasn't enough), so I out for alternatives, and found quassel. But I hated the looks of Quasseldroid, so I forked it, and started working on the UI, and on features by reversing the protocol yet again. Not having actually programmed anything except for some delphi and VB.NET stuff, a tiny java project and one C# Windows Phone 7 app, my code was the worst, ever. Seriously, it was bad. After a while, discussion came about about turning this code into a PR, so I threw it all away, and rewrote it again, still bad, but it worked. This was merged, I became maintainer of Quasseldroid (because no one else was working on it anymore), and then, around 2015, I reversed the protocol, read the entire source of every implementation, wrote it all down on paper, studied every file format and every quirks and then I rewrote quasseldroid from scratch, in about 3 months, with every feature of the desktop version. I called this The Next Generation of Quasseldroid, jokingly quasseldroid TNG or later quasseldroid-ng.
A few weeks later, a new Android version came out, introducing Doze, and breaking everything about quasseldroid-ng. And so I rewrote it again, and before release, a new Android version broke it all again.
And that basically repeated, until I decided that it can't continue like this, and if we'll do major changes to the protocol, we might actually get a working version for Android that lasts longer than a few weeks in Beta before Google breaks it. So I learnt C++, and started contributing.
And that is basically my view of the story. Reverse engineering a protocol again and again, seeing variations of variations of the same protocol everywhere used, never properly documented. The Qt documentation is entirely different from what Qt actually puts on the wire. Blobs in the database.
But it also means backwards compatibility in the desktop client/core for every version from the past 10 years, and backwards compatibility on Android for every version of the past 6 years.
And now, maybe, I'll be able to help replace this, bit by bit. After the improvements to tge protocol that added major performance benefits, the next part is replacing the bouncer-side config format entirely, so I can properly containerise it.
TL;DR: no matter how good the support for a non-standard binary serialization format in your favourite language is, 10 years down the road people will reuse your protocol in a dozen more languages, and they'll have to reverse the protocol themselves, and will do a semi-good job at it, and because you never thought about backwards compatibility you now have a mess (we had luck because 99% of what we transmitted were key/value maps, and when reading we always used default values if the key didn't exist and ignored unused keys. Sometimes we did serialize structs, basically, though, and those places still cause me headaches today, and require workarounds to add features, e.g. the latest sendermode implementations)
Re: Ask HN: Is big-endian dead?
#153> Little-endian is slightly more confusing for humans... I think you meant to write “Little-endian is slightly more confusing for humans who use left-to-right languages ” as all the R-L languages also put the least significant digit on the right.
from a bytes as representing a bitstring perspective, big endian make sense, and little endian is a terrible scramble.
Re: Ask HN: Is big-endian dead?
#154Eh, haswell added the movbe instructions. And I wouldn't be surprised if someone in the RISCV world added big endian instructions (they're nice for network processing). And nearly every powerpc I've seen is big endian, FWIW.
(disclaimer: IBMer)
Re: Ask HN: Is big-endian dead?
#155Little-endian is slightly more confusing for humans I've heard this before, but the reason is that you view hex data and list numbers left-to-right as if they were letters. They are not. 0x12345678 stored big-endian, numbering bytes left-to-right: 12 34 56 78 Looks good, but I think that this is actually more confusing, because when you number the bytes and bits you will see that the bytes are written left-to-right,…
At least in Freescale's PowerPC documentation, it's convention to number the bits left-to-right in big-endian. So the most-significant bit is bit 0, which matches up with the most-significant big-endian byte being 0. See, for a random example, page 1101 of https://www.nxp.com/docs/en/reference-manual/MPC8379ERM.pdf . Personally I prefer the little-endian representation.
The main practical problem with MSB0 is that you need to consider the width of whatever field or register you're looking at to work out the correct bit shift.
Re: Ask HN: Is big-endian dead?
#156Earlier quoted context omitted.
Not exctly accurate, for the part of the number that is Also for the part that is L-R, that is not the rule, as some people still read all the number as R-L (actually in a lot of historical documents that was the case), so the would read 1925 as five and twenty and nine hundred and a thousand. Where is now most people would read it as a thousand, and nine hundred, and five and twenty.
Many R-L languages do the same such as German French, Hindi et al.
Source: native Hindi speaker.
Re: Ask HN: Is big-endian dead?
#157Earlier quoted context omitted.
Not exctly accurate, for the part of the number that is Also for the part that is L-R, that is not the rule, as some people still read all the number as R-L (actually in a lot of historical documents that was the case), so the would read 1925 as five and twenty and nine hundred and a thousand. Where is now most people would read it as a thousand, and nine hundred, and five and twenty.
Many R-L languages do the same such as German French, Hindi et al.
Re: Ask HN: Is big-endian dead?
#158Re: Ask HN: Is big-endian dead?
#159Earlier quoted context omitted.
> Assuming little-endian CPU arch. It's followed by a byte reorder on big-endian architectures. Basically all common CPUs are LE. (And basically all BE CPUs have dedicated instructions for reading LE data. It's true I haven't yet added the inline assembly to use those instructions in Cap'n Proto's reference implementation, but that's only because no one actually cares about these architectures.) So in basically all r…
> but that's only because no one actually cares about these architectures. This. Optimizing performance on BE is a waste of everyone's time and resources.
Re: Ask HN: Is big-endian dead?
#160Big endian is dead on the client. WebGL exposes endianness and happened at a time when almost all systems running browsers are little-endian. This makes the big-endian provisions of the spec a dead letter. Web devs don't need to test on big-endian systems and would have a hard time finding big-endian systems to test with even if they wanted to. It's safe to assume, therefore, that there's a mass of Web content that o…
TenFourFox and Leopard Webkit both emulate little-endian typed arrays on BE Power Macs and transparently byteswap integers. This mostly works for sites that care and has almost no overhead in JITted code for 16 and 32-bit int, but Facebook managed to break some of our assumptions with floats (example: https://github.com/classilla/tenfourfox/issues/453 -- we don't byteswap floats because it's expensive and probably would break things expecting native endian on the DOM side). Fortunately, TenFourFox doesn't support WebGL, so we don't need to worry about that.
IMHO, this is a broken promise about how the Web was supposed to be platform-independent. I get the feeling the general attitude on this thread is that such platforms are unimportant, but that doesn't mean it's not a broken promise.