Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's TCP vs. RPC All over Again

systemsapproach.substack.com

Post reply on HN
Print view View original

51–60 of 116 posts

Re: It's TCP vs. RPC All over Again

#51

Post by cryptonector »

cjsplat wrote: post #33
cryptonector wrote: post #13

Earlier quoted context omitted.

There's so many... Apollo Domain's. ONC RPC. DCE RPC (and MSFT RPC, which derives from DCE RPC). SOAP. LustreRPC. And who knows how many others. The biggest problem with RPC in general is that historically it was synchronous because that's what was easy to implement in the 80s. Fix that and an RPC sucks only as much as the encoding system it uses. But RPC == remote procedure call, and that causes people to instantly…

Well of course once you say "procedure call", people think synchronous. For the sender, asynchronous RPC is just a convenient marshalling and return interface for the sender. It is pretty clunky for the A-RPC caller to create an illusion in the code of the RPC being "just like" a local procedure call. For the called procedure an Async RPC can look just like a local sync procedure call, except for that whole address s…

> I think it was popular because it was easy to code to.

That's not that unlike it being easy to implement.

> Multi-threaded coding is easy to get wrong, [...]

Yeah, but the better design is async/await and so on, not threading, but that stuff was not very popular in the 80s.

The approach to concurrency in the 80s was heavy on context switching because that's what was inherited from the 70s. It feels like the thinking was along the lines of "during the time you're waiting for a response some other process will run, and since we're used to heavy latency, who cares!", but that became less and less tenable. There's a reason all the distributed operating system research of the 80s died except for filesystems -- with filesystems you can consciously avoid hot ping pong by just not accessing shared resources concurrently, while the filesystem itself remains super useful.

Re: It's TCP vs. RPC All over Again

#52

Post by matjet »

codeflo wrote: post #21
manv1 wrote: post #19

This paragraph is just wrong: "Another explanation is that the Internet has unnecessarily coupled the transport protocol with the rest of the RPC framework. Conflating the two naturally follows from the purpose-built examples I just gave: SMTP is bundled with MIME; SNMP is bundled with MIB; and HTTP is bundled with HTML. " I think at this point he's suffering from what old-timers used to call recto-cranial inversion.

I think I feel dumber for reading this. (The referenced paper by Ousterhout is quite good though, and doesn’t confuse the terminology. Maybe that one’s what HN should link and discuss.)

To someone only weakly aware in this area, can you explain what is wrong with the statement? From a cursory view, these protocols indeed seem to normally carry the mentioned payloads.

Re: It's TCP vs. RPC All over Again

#53

Post by kwantam »

Is this a good time to bring up T/TCP?

https://www.rfc-editor.org/rfc/rfc1644

    This memo specifies T/TCP, an experimental TCP extension for
    efficient transaction-oriented (request/response) service.  This
    backwards-compatible extension could fill the gap between the current
    connection-oriented TCP and the datagram-based UDP.
I have no illusions about the likelihood it'll ever see use, but I've always thought T/TCP was cool and fun.

Re: It's TCP vs. RPC All over Again

#54

Post by cryptonector »

manv1 wrote: post #19

This paragraph is just wrong: "Another explanation is that the Internet has unnecessarily coupled the transport protocol with the rest of the RPC framework. Conflating the two naturally follows from the purpose-built examples I just gave: SMTP is bundled with MIME; SNMP is bundled with MIB; and HTTP is bundled with HTML. " I think at this point he's suffering from what old-timers used to call recto-cranial inversion.

Agreed. MIME came after SMTP, for example, and HTTP is used for all sorts of not-HTML-or-hyper-anything stuff. That statement smacks me of "wow, HTTP is a pretty good RPC, but too bad it's only/mainly for transporting HTML, shucks!". The name of the protocol is not very representative anymore either of how it's used or how anyone should use it. If HTTP fits the bill, then use it and move on.

Re: It's TCP vs. RPC All over Again

#56

Post by kwantam »

cjsplat wrote: post #41
roncesvalles wrote: post #37

You could just use UDP instead of TCP and implement some subset of TCP features that you need in-band, like sequence numbers. This is not a new concept. The Facebook Memcache paper [1] in 2013 described using UDP for intra-datacenter requests. [1] https://research.facebook.com/publications/scaling-memcache-...

Sure, but what is UDP bringing to the party then? The pseudoheader? A reasonable RPC layer could just layer on IP.

Well, for one, UDP brings the U. It's called the User Datagram Protocol because (at least on UNIX-like systems) unprivileged programs can almost always send and receive UDP datagrams, but in general cannot send and receive raw IP datagrams.

Re: It's TCP vs. RPC All over Again

#57

Post by deathanatos »

hardwaresofton wrote: post #29

If you'd like to watch Ousterhoust explain Homa and his vision of replacing TCP: https://www.youtube.com/watch?v=o2HBHckrdQc Spoiler: Ousterhout is right, and the data shows he is right. Maybe not everyone knows it yet, or it's the kind of "right" that not everyone will ever get to (and we'll settle for "good enough" for much longer), but he is almost certainly right. Not to disparage the poster of this article of co…

I think you're right that he's "right", but the writing is shooting itself in the foot here. I agree that there's a mostly-unexplored middle between TCP & UDP, but you need to not utter things like "HTTP is bundled with HTML". I also think RPC is a poor label for "request/response" protocol (HTTP is request/response, and many of the problems here apply to HTTP too … but it isn't RPC).

I'd like more discussion of the contenders in the space: why do they not work? QUIC is barely mentioned beyond "we tried it and nope". SCTP isn't mentioned on the OP at all, but if you click through the Homa stuff a bit, it's written off as "a WAN protocol" — whatever that is? I get that his focus is datacenters, but "WAN protocol" is what I feel most of us are interested in, and I don't immediately see why WAN vs. DC matters¹.

What I'm also not getting from the Homa page is … what is the protocol, what are it's packet structures? Is there no documentation? (I've not hit all the third-degree links because I'm not a spider, but nothing seemed relevant or it seemed a little relevant but didn't hold the answers.) The old adage about data structures over algorithms seems to apply:

> "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." -- Fred Brooks, The Mythical Man Month (1975)

¹I'd also want to know why a good "WAN" protocol that solves the stated problems wouldn't equally apply to the datacenter, modulo needing some adjustments around assumptions made about the network's latency and maybe reliability: a DC network, I'd hope, is going to have much lower latency. But like Raft timers, it seems like a by-use-case adjustable constant.

Re: It's TCP vs. RPC All over Again

#59

Post by aidenn0 »

Osterhout's response to Pepelnjak's criticism is solid, but he mentions not being familiar with SCTP. Homa is much more fairly compared to SCTP than TCP, since there is a large intersection with the problems they solve. Osterhout seems to think that ACK-based sender congestion avoidance is a significant problem in the datacenter.

SCTP uses ACK-based sender congestion avoidance, but lacks the head-of-line blocking that TCP has[1]. Comparing Homa to SCTP would be a good way to test this thesis.

1: Or rather it's optional.

Re: It's TCP vs. RPC All over Again

#60

Post by pavlov »

danans wrote: post #46
Animats wrote: post #30

There have been lots of RPC protocols. Here are some still in use. Transport level: * Sun RPC [1]. QNX still uses this. It can run over UDP or over raw Ethernet. It just transfers an array of bytes and gets an array of bytes back - marshalling is a higher level problem. It handles messages bigger than one packet, and retransmission. It's simple and performance is good, but there is no security. * Stream Control Trans…

> CORBA - you define interfaces in a special language and compile them. Data is not self-describing. OMG the Common Object Request Broker Architecture (which coincidentally was created by OMG). I never thought I'd see that acronym again, and all these years later, I still don't understand what it is.

But is it not obvious? It’s the architecture for brokering requests of objects most common.
Post reply on HN