> You want individual messages instead of a byte-stream? TCP has an option for that.
That links to. https://book.systemsapproach.org/e2e/tcp.html#record-boundar... In my opinion you can't be serious suggesting usage of URG or PUSH as message boundaries. TCP does not work with records/messages. It works with streams. There is a reason for that.
The reason is that a record can be large. If it fits one packet - who cares. With small records maybe the transport protocol can concatenate/batch the records to save the network some load (Nagle's algo). With large records, you need to do retransmission if data is lost, you have head-of-line blocking, you must deal with congestion backoff, flow control (what if remote end is busy, or has scarce memory).
In traditional IP the solution to RPC would be to open a new stream for each request. This is how HTTP 1.0 worked. It had certain advantages.
> You want congestion control? TCP can give you one version tuned for the wide-area and another version tuned for the datacenter.
You can go quite far without tuning congestion control algos inside datacenter. Cubic and BBR are good enough for almost everyone. There is cost in using DCTCP. It's in the same ballpark as ECN. Great in theory, solves real issue. But requires heavy investment to get any value off it.
> You want a low-latency network stack? Well that’s a challenge TCP has a 40-year history of trying to optimize away, and when that falls short, ultimately looking to SmartNICs to solve.
Okay, yes, TCP was not intended to be low latency. With traditional API's it's impossible to get zero-copy, gosh, even stuff like getting a transmission completion signal is basically impossible. Normal NIC's have offloads and they work, they save the CPU from some dumb work. If you want low latency then go for RDMA-like approach.
> In contrast, RPC was designed from the start to optimize round-trip performance in low-latency networks.
Err... Okay, so by this definition HTTP is not RPC. Fine.
Okay, so we're talking about inter-datacenter "trusted", "fast", "low latency", with homogenous network, protocol, that is tuned to RPC-style, request-response traffic. Stuff like this comes to mind: Memcached (including binary protocol), redis, RDMA, grpc, and quic.
Fine! TCP is indeed not optimal. Should you care? Nope. In real life tricks like tcp connection-pooling, raw UDP protocols (think: memcached UDP) work just fine.
Furthermore, often you need encryption (even inside datacenter). The power of TCP and other generic protocols is that they work fine on the lossy, untrusted public internet. I can connect to redis over 300ms lossy stream just fine (and I often do!).
Would we benefit if there is a protocol (like QUIC or SCTP) that is tuned for inter-datacenter RPC - probably yes. Should you care - nope. Look at how hard making QUIC was.
I'm a pragmatist and I'm allergic to theoretical discussions. There is a reason why memcached binary protocol and memcached UDP protocols are super obscure and (almost) nobody runs them (the reason is that latency is not the most important thing. Simple code in the client is often more important).
> But coming back to the specific question of RPC vs TCP in the datacenter, it still has me scratching my head about why it hasn’t happened
I can give one answer: BSD sockets API. They are limiting, and think about a case of large response. Ideally the server would like to give it to the kernel and move on to next request. This is not how the API's work. I think SCTP + some kind of zercopy would give you quite a decent starting point. However, SCTP didn't catch on, and the API has serious flaws. The next big thing is QUIC. But unless someone provides a kernel API for it, it won't be "fast" or "low latency".