HTTP-style query strings are a horrible format, whether you like ASCII or not.
Designing Network Protocols
11–20 of 24 posts
Re: Designing Network Protocols
#12I think the short version is: in the past 5 years, simple wire-oriented serialisation formats have become much more common. At the time you had your pick of ASN (humongous) or Thrift (brand new).
Not particularly fair. You also had XDR (ubiquitous on Unix systems), IIOP, TLV, ICE... not to mention what ever protocol designer for the past 20 years has used: network byte order integers and ASCII/UTF8 strings. Some people just like ASCII, human readable protocols. There's nothing wrong with that, but it's a little silly to suggest that the options for a packed binary encoding in 2007 were limited because Thrift…
You're right, to my shame. I should've thought of XDR myself, after suffering through a lecture on it at uni.
I'd be interested in the original designer's remarks on using existing wire serialisation formats.
Re: Designing Network Protocols
#13Earlier quoted context omitted.
So the network protocol must be in text so that a network admin could debug it? This is absurd.
How does a network admin debug a binary protocol for which no dissector has been implemented/merged into core for Wireshark, and no decoder has been written for tcpdump? It's obviously doable, but it's very painful.
On the other hand, are Wireshark and tcpdump now the gatekeepers for new protocols?
Re: Designing Network Protocols
#14Earlier quoted context omitted.
How does a network admin debug a binary protocol for which no dissector has been implemented/merged into core for Wireshark, and no decoder has been written for tcpdump? It's obviously doable, but it's very painful.
On the other hand, are Wireshark and tcpdump now the gatekeepers for new protocols?
What's your point? I'm not making a value judgement.
Re: Designing Network Protocols
#15Earlier quoted context omitted.
So the network protocol must be in text so that a network admin could debug it? This is absurd.
How does a network admin debug a binary protocol for which no dissector has been implemented/merged into core for Wireshark, and no decoder has been written for tcpdump? It's obviously doable, but it's very painful.
Isn't Wireshark extensible in Lua?
I can see both sides of the argument here, but basing a protocol on text just for the ease of eyeballing it on-the-wire seems like optimizing for the uncommon case.
Heck, almost any decent protocol should only have ciphertext on-the-wire anyway.
Re: Designing Network Protocols
#16Earlier quoted context omitted.
How does a network admin debug a binary protocol for which no dissector has been implemented/merged into core for Wireshark, and no decoder has been written for tcpdump? It's obviously doable, but it's very painful.
Isn't Wireshark extensible in Lua? I can see both sides of the argument here, but basing a protocol on text just for the ease of eyeballing it on-the-wire seems like optimizing for the uncommon case. Heck, almost any decent protocol should only have ciphertext on-the-wire anyway.
That's more or less like saying "well they can just write the decode". They're network administrators. If you use an ASCII protocol, they don't have to do anything.
Re: Designing Network Protocols
#17Earlier quoted context omitted.
On the other hand, are Wireshark and tcpdump now the gatekeepers for new protocols?
What's your point? I'm not making a value judgement.
You say that to me a lot.
My point is that I imagine a network designer shouldn't focus on Wireshark or tcpdump integration over other non-functional requirements such as, well, network performance.
Network performance isn't as visible as the non-functional requirement of inspectability because it is amortised over potentially millions of machines, whereas inspectability is an immediately visible issue to the select few who "pop the hood" to fix an issue or simply to have a look.
For example: in terms of network capacity, I wonder how much HTTP headers cost all of us collectively. Probably a lot more than the cost of making a Wireshark plugin and having sysadmins install it as necessary.
Edit: put another way, I think designers should prioritise the needs of the people who pay the cost of network operation over the convenience of the operators.
There's a feedback loop here -- if it's too hard and thus very expensive to operate a system, then optimising for performance was a false win. But I don't think this is such a case, especially since as you pointed out elsewhere there are a number of very mature binary wire formats that were extant in 2007.
Re: Designing Network Protocols
#18Earlier quoted context omitted.
What's your point? I'm not making a value judgement.
You say that to me a lot. My point is that I imagine a network designer shouldn't focus on Wireshark or tcpdump integration over other non-functional requirements such as, well, network performance. Network performance isn't as visible as the non-functional requirement of inspectability because it is amortised over potentially millions of machines, whereas inspectability is an immediately visible issue to the select…
See: http://cr.yp.to/sarcasm/modest-proposal.txt
"I implore [you] to remember Dave and Virginia, preying on the drug addicts of the next generation and the sexually dissatisfied men of the previous generation. How different their careers could have been if their parents had not downloaded so many terabytes of data! We must not abandon our children to such a fate."
Re: Designing Network Protocols
#19Earlier quoted context omitted.
Isn't Wireshark extensible in Lua? I can see both sides of the argument here, but basing a protocol on text just for the ease of eyeballing it on-the-wire seems like optimizing for the uncommon case. Heck, almost any decent protocol should only have ciphertext on-the-wire anyway.
That's more or less like saying "well they can just write the decode". They're network administrators. If you use an ASCII protocol, they don't have to do anything .
I'm saying someone can write the decode and share it on their blog post or Github and your admin can start using it without having to recompile Wireshark. (I think, haven't actually tried it myself).
But even still, this only matters if:
A. The protocol is so new that Wireshark isn't shipping a parser,
B. the admin's stuff isn't working,
C. the admin can't get his stuff working by normal troubleshooting and must resort to observing the protocol,
D. the admin can't get his stuff working by observing the binary representation of the protocol, and
E. the admin actually can get his stuff working with a transliterated ASCII representation of the protocol.
Certainly I would probably find it easier to troubleshoot a text-based protocol too. I just think it's a relatively minor case in the grand scheme of things.
Re: Designing Network Protocols
#20I would accept the ease of debugging argument if these messages weren't so small and so common. 1000 servers constructing strings and sending them over the network once a second is a nontrivial waste of resources.