Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to design a wire protocol

esr.ibiblio.org

Post reply on HN
Print view View original

61–70 of 71 posts

Re: How not to design a wire protocol

#61

Post by leoc »

DonHopkins wrote: post #51
outworlder wrote: post #42

Earlier quoted context omitted.

Not terrible. I have learned some things about how the NTP protocol was designed. And it is a good summary of trade-offs involved. You may or may not agree with them, and they may or may not be important to you. However, I have learned even more things about how absolutely mind-bogglingly huge the egos of some posters here are.

Funny that you should mention mind-bogglingly huge egos in a discussion about ESR: > “Are you” she asked “the most famous programmer in the world?” > This was a question which I had, believe it or not, never thought about before. But it’s a reasonable one to ask... http://esr.ibiblio.org/?p=6839

What's the relevance of this? It's certainly not the case that the presence of one person with an embarrassingly obvious craving for adulation in the conversation means that other people with distended egos of their own will stay away. The opposite is nearer true.

Re: How not to design a wire protocol

#62

Post by DonHopkins »

leoc wrote: post #61
DonHopkins wrote: post #51

Earlier quoted context omitted.

Funny that you should mention mind-bogglingly huge egos in a discussion about ESR: > “Are you” she asked “the most famous programmer in the world?” > This was a question which I had, believe it or not, never thought about before. But it’s a reasonable one to ask... http://esr.ibiblio.org/?p=6839

What's the relevance of this? It's certainly not the case that the presence of one person with an embarrassingly obvious craving for adulation in the conversation means that other people with distended egos of their own will stay away. The opposite is nearer true.

If you're so sure that the grandparent comment is relevant, then please tell us exactly which messages you agree are from people with absolutely mind-bogglingly huge egos even larger than Eric S Raymond's? I haven't noticed anyone making any appeals to their own authority as arrogant as claiming to be the most famous programmer in the world. Are you saying that it takes a huge ego to point out ESR's long track record of making flawed arguments, and quote the many patently false, self-aggrandizing, racist and sexist things he's said in the past, or that there's something wrong with pointing out racism instead of silently condoning it?

Re: How not to design a wire protocol

#63

Post by richardwhiuk »

DoofusOfDeath wrote: post #12

IIRC, Silicon Graphics (the company) used to maintain a registry for "magic numbers" designating each file's type. Customers could contact SGI to be issued a magic number for whatever file formats they were cooking up. I wonder if there's merit to recreating such a thing under ICANN, where the issued serial numbers are useful for file types, wire protocols, etc. Then anyone needing to reliably interpret a packet coul…

https://github.com/file/file/tree/master/magic/Magdir exists for file types.

Re: How not to design a wire protocol

#64

Post by deathanatos »

altfredd wrote: post #58
deathanatos wrote: post #36

Earlier quoted context omitted.

> And thus JSON can't... […] properly encode non-Latin text (no, — hexadecimal encoding is even worse than no encoding); This is (RFC) valid JSON: "我喜欢吃鱼。" Sure, JSON has some corner cases. Binary protocols can, and do, as well. While I'm sure that non-compliant JSON examples exist in the wild, I would think that overall they're exceedingly rare compared to compliant ones. And if you don't like the limitations of JSO…

> This is (RFC) valid JSON: You mean, "valid, according to the latest 2017 RFC". Such young RFC is still to raw, too immature to adopt, especially if it concerns data interchange formats. IPv6 was created in 1995, and it apparently still too young! I fear, that a proper full-featured JSON spec, with comment support, mandatory UTF-8 and strict prohibition of hex-encoding won't be created and implemented by most JSON p…

> You mean, "valid, according to the latest 2017 RFC". Such young RFC is still to raw, too immature to adopt, especially if it concerns data interchange formats. IPv6 was created in 1995, and it apparently still too young!

No, I mean valid, according to the oldest, 2013 RFC and all later standards. Non-ASCII characters, encoded directly w/o escaping, have always been supported by JSON. (JSON comes from JavaScript's syntax, and it's legal there, too.)

> I fear, that a proper full-featured JSON spec, with comment support

Many of us use JSON as a language to exchange data, service to service. Comments do no good in that regard. JSON, even w/ comments, is not terribly friendly. I'd recommend TOML or YAML, depending on the situation.

> mandatory UTF-8

JSON is required to be encoded in one of the Unicode UTF encodings. So, it's not required to be UTF-8, but it's pretty close, and I don't think I've yet run across a JSON document that wasn't UTF-8.

> strict prohibition of hex-encoding

I don't think you'd really want this. (Particularly if you want human-friendly features, like comments…) In debug situations, certain non-printing characters are just easier to deal w/ if they're not printed, for example.

Re: How not to design a wire protocol

#65

Post by bsder »

robertAngst wrote: post #33
bsder wrote: post #14

If one of my engineers brought me this "protocol" (um, protocols require state machines, where is it?), we would be having a series of long talks to figure out whether he needs to be educated or fired. What happens if I want to run NTP on an ARM M4 microcontroller with a lithium coin cell battery? Because, you know, perhaps I actually might want my time to be accurate on devices that even outship cell phones? Sending…

Why would someone write a protocol in 2019? Don't we already have formats? (EDIT: Like CAN or I2C?)

Don't know why you got downvoted, it's a good question.

A "protocol" sits on top of things like I2C, SPI, and CAN.

Protcols answer things like: "How do I send more bytes than the underlying transport can take in a single transaction?" "How do I exchange data when hardware has different characteristics." "How do I minimize the power or time needed to exchange data?"

Different protocols have different strengths and weaknesses.

Remember: part of my complaint about this "protocol" is its "verbosity". If you are on a battery or are bandwidth constrained (Narrowband-IoT, LoRA, ANT, etc.), you want a protocol that exchanges short messages. Time is certainly something that you don't want to require lengthy messages when you are trying to set up.

Too many people think "embedded" means "runs a Linux installation larger than the average computer in 1996".

Re: How not to design a wire protocol

#66

Post by bsder »

outworlder wrote: post #41
bsder wrote: post #14

If one of my engineers brought me this "protocol" (um, protocols require state machines, where is it?), we would be having a series of long talks to figure out whether he needs to be educated or fired. What happens if I want to run NTP on an ARM M4 microcontroller with a lithium coin cell battery? Because, you know, perhaps I actually might want my time to be accurate on devices that even outship cell phones? Sending…

> Please quit giving ESR a platform when it's quite clear he really sucks as a programmer. I am not sure this post complies with Hacker News guidelines.

Why? An assessment of technical competence/relevance seems to be a cornerstone of the upvote/downvote system on HN.

I find ESR to be ferociously overrated from a technical standpoint and resent the fact that he absorbs oxygen from people far more talented but far less "adept" at self-promotion. In addition, the "technical" ideas that he promulgates occasionally have to be actively undone by those with stronger technical chops.

Why does verbalizing this run afoul?

Re: How not to design a wire protocol

#68

Post by _57jb »

axaxs wrote: post #13
notacoward wrote: post #9

When the very first sentence is incorrect, the outlook for the rest looks grim. > A wire protocol is a way to pass data structures or aggregates over a serial channel between different computing environments. A real wire protocol involves not just the flow of data but the flow of control. Without knowing which data structures are replies to which others, issued under what circumstances and affecting which other parts…

How dare you insult the author of one quasi used json format!! I don't like the article for the same reasons. It felt long-winded and didn't ultimately offer anything of value. What's worse is that it's written from a position of authority, common for this particular author, so some people will be duped into treating it as gospel.

So glad to see this comment as I felt exactly the same way.

I would look forward to reading articles from any point of discussion however strongly people felt and he seems like he would make for a great debate but at the end it was...."You think you know but you don't."

Seems like a classic ivory architect mentality that grinds potentially good conversations to a halt.

Re: How not to design a wire protocol

#70

Post by hedora »

wmf wrote: post #8
alexeiz wrote: post #4

Earlier quoted context omitted.

I was a bit surprised myself that the author prefers JSON to bit-packed protocols, but given that the author is none other than Eric Raymond, I have to take him seriously. Accidentally, in my work lately I used JSON for data exchange over the network where performance is not important, and MsgPack otherwise where it is (which is essentially a packed JSON).

There's no need to appeal to authority; the article gives specific reasons when to use JSON and when to use binary. What do you think about those arguments?

There’s a nice rebuttal from one of the ntp people in the article’s comments.

JSON is a disaster for many reasons. Hardware incompatible floating point is one; inconsistency in parser implementations (and ambiguities in the spec) also don’t help.

Also, why use a tree structured data representation when the underlying data structure is fundamentally just a N-tuple with a fixed schema?

Similarly, why use a text protocol to send around fixed length blobs or encrypted data?

Post reply on HN