Ask HN: Is big-endian dead? • Hacker News Board

Re: Ask HN: Is big-endian dead?

#191

Post by kentonv » Sun, Jan 21, 2018, 11:33 PM UTC

kuschku wrote: post #170

kripke wrote: post #164

Earlier quoted context omitted.

This is interesting, and thanks for trying to clean up the protocol, but isn't this an orthogonal issue? The problems you are describing seem to be related to using an undocumented protocol, which is unrelated to using a custom serialization format vs building upon an existing one. Building upon an existing, well-documented, and relatively sane serialization format (protobuf, capn't proto, message pack, json, heck ev…

> Building upon an existing, well-documented, and relatively sane serialization format The problem with that is that whatever format seems well-documented and relatively sane today might become an obscure, unknown protocol 10 years down the road.

FWIW, Protobuf has now been open source for a decade and has been used for basically everything inside Google since about the turn of the century. Protobuf predates JSON, and I would wager that, worldwide, much more data is stored in Protobuf format and many more cycles are spent parsing Protobuf format than JSON. For Protobuf to die out, Google itself would have to die, as would quite a few other companies that heavily rely on it. It doesn't seem likely to happen any time soon.

I unfortunately am not in a position to make such strong statements about Cap'n Proto. However, implementations exist in C++, Java, JavaScript, Rust, Go, Python, and a bunch of other languages, so it should at least be much easier to deal with than Qt serialization.

(Disclosure: I'm the author of Cap'n Proto and of the first open source release of Protobuf.)

Re: Ask HN: Is big-endian dead?

#192

Post by astrodust » Mon, Jan 22, 2018, 1:43 AM UTC

asterius wrote: post #187

tlb wrote: post #103

Earlier quoted context omitted.

But more than that, the encoding used in 10GBASE-T doesn't allow decoding individual bytes. For the usual copper standard, it's 4 pairs with 7 bits per symbol, and the skew between lanes means you can't count on corresponding symbols arriving at the same time. And then there's a reed-solomon block code to correct errors, which has to be decoded all at once. The optical standards use 64b/66b coding, which means you re…

No one who cares about latency even has 10GBASE-T hardware, because it introduces a vast amount of coding latency. It is also very power inefficient.

Right, it's probably even "chatty", too and "won't scale".

Re: Ask HN: Is big-endian dead?

#193

Post by nostrademons » Tue, Jan 23, 2018, 12:51 AM UTC

kentonv wrote: post #191

kuschku wrote: post #170

Earlier quoted context omitted.

> Building upon an existing, well-documented, and relatively sane serialization format The problem with that is that whatever format seems well-documented and relatively sane today might become an obscure, unknown protocol 10 years down the road.

FWIW, Protobuf has now been open source for a decade and has been used for basically everything inside Google since about the turn of the century. Protobuf predates JSON, and I would wager that, worldwide, much more data is stored in Protobuf format and many more cycles are spent parsing Protobuf format than JSON. For Protobuf to die out, Google itself would have to die, as would quite a few other companies that heav…

To be fair, DEC was once in the same position as Google; in fact, by employee count, it was twice as big (140k vs. 70k) and by market share of the whole computing market (you could speak of a "computing market" back then), it was significantly larger. In the mid-80s, the idea that a VAX might be supplanted by a massive worldwide computation network of billions of computing devices would've seemed like science fiction. (Note that at its peak, Digital had only sold 400,000 VAX.) You could be fairly confident that storing your data in the OpenVMS filesystem would be fairly future-proof.

When was the last time you saw a filename of the form NODE"accountname password"::device:[directory.subdirectory]filename.type;ver?