Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a replacement for TCP in the datacenter [pdf]

web.stanford.edu

Post reply on HN
Print view View original

261–270 of 329 posts

Re: We need a replacement for TCP in the datacenter [pdf]

#261

Post by tristor »

nickdothutton wrote: post #208

23 years ago I sat in a meeting with Sun, Intel, Mellanox, and 1 or 2 others. In that meeting we discussed putting an RDMA interface on individual hard drives, trays of RAM, CPUs, and other more exotic devices (like battery backed RAM, no conventional SSD in those days of course). You’d install RAM 1 42U rack at a time, disks likewise, CPUs in another rack and so on. All partitioned, controlled, managed, and of cours…

I was in the room for similar discussions as part of the OpenCompute project, which was around remote IO and resource disaggregation. There are systems like this, and they may make sense for certain use cases, but generally speaking hyperscalers today are built around virtualization which doesn't align well to this model.

Re: We need a replacement for TCP in the datacenter [pdf]

#262

Post by cryptonector »

josephg wrote: post #158
cryptonector wrote: post #128

Earlier quoted context omitted.

> The only kinda-sorta exceptions I know about are HTTP/1.1 and telnet. But HTTP/1.1 is still a message oriented protocol; just with file-sized messages. (And even this stops being true with http2 anyway). No, HTTP/2 and QUIC do not change the semantics of HTTP. Also, you can have endless streams with HTTP/1.1: just use chunked encoding to POST/PUT and use Range: bytes=0- and chunked encoding for GET and chunked enco…

Correct me if I’m wrong, but doesn’t h2 still break up requests and responses into smaller message frames in order to do multiplexing? Those message frames are what I’m talking about - as I understand it, they are, yet again, a message oriented protocol layered on top of tcp.

You will always need _some_ framing if you're dealing with bulk data.

If it's not bulk, then you don't need framing if everything can fit into one datagram / whatever transfer unit provided by the transport, but the transport itself will need some framing, especially if it will need to support any kind of fragmentation.

If it's not bulk and it doesn't fit in a datagram / whatever transfer unit provided by the transport and the transport doesn't do fragmentation, then you have to do framing yourself, and then you have a sequencing problem, and so on, and you quickly re-invent parts of TCP but at the application layer.

Basically, it seems inescapable that the Internet is based on packets, and that packets are limited in size, and so application protocols have to be smeared onto packets.

Things are only ever trivial when you're doing request/response protocols with always- or mostly-small requests and responses. The moment you need anything that doesn't fit in the path MTU minus overhead, you need framing.

So I don't think that an octet stream abstraction is quaint and obsolete.

Re: We need a replacement for TCP in the datacenter [pdf]

#263

Post by JoeAltmaier »

delroth wrote: post #236
JoeAltmaier wrote: post #227

Google hasn't used TCP in the datacenter for years. What they use I don't know. But it's even custom switches with custom chips. My son did work in graduate school for a clean-slate network implementation of a network for the datacenter. Maybe Google, I don't remember. One issue I remember they addressed was, scheduling bandwidth for VM migration within their datacenter cloud. See, some customer reserves a 'machine'…

> Google hasn't used TCP in the datacenter for years. That's absolutely false. I don't have any sources except for having worked at Google from 2013-2022, but it's not like you quoted any sources either, so... There's a reason why Google is still releasing stuff like TCP BBR (2017).

Well, the first link googling 'google datacenter hardware' is google's article on how they don't use standard TCP hardware or software in their datacenters. But I guess that was too much to ask...

Re: We need a replacement for TCP in the datacenter [pdf]

#264

Post by eigenrick »

lizknope wrote: post #215

For those unfamiliar with the author. https://en.wikipedia.org/wiki/John_Ousterhout He is probably most famous for having created the Tcl language and Tk GUI library. He also worked on the Sprite distributed operating system, the Magic VLSI design tool, and a bunch of other things.

Also, more recently, one of the inventors of the RAFT algorithm.

Re: We need a replacement for TCP in the datacenter [pdf]

#265

Post by jmull »

eigenrick wrote: post #254
jmull wrote: post #247

I get the issue with TCP, but I'm not sure about Homa... e.g., why not UDP with some homa-like semantics on top? That might really ease the "Getting there from here" issue. In fact, what's actually being suggested is for applications to replace calls to TCP-based APIs with calls to gRPC APIs (or other high-level RCP APIs), where the transport layer becomes an implementation detail. Fair enough, but this is a very rou…

It would be interesting to see a write-up from them on Homa's benefits over UDP. UDP has been used to work-around issues with TCP both in the datacenter and in unreliable WANs. Skimming over the paper, I think the magic of Homa is in its RPC calls and its short-lived connections. When they're handled at layer 3 and 4, they can provide a significant hint to switches, routers and hosts regarding prioritization and cong…

Here's what I'm thinking:

You add a homa-like header inside a UDP packet. Inside the data center you use switches, nics, etc that know and understand the home-like protocol and can implement homa-like behavior... as needed. Anywhere else, you can fall back to UDP, due to its ubiquity.

Yes, various things would have to somehow know the homa-like protocol was being used... just like various things would have to somehow know the homa protocol was being used. Yes, different vendors would have to have compatible homa-like implementations... just like different vendors would have to have compatible homa implementations.

I think the complexity you mention is inherent in anything that actually gets more widely deployed and used.

Re: We need a replacement for TCP in the datacenter [pdf]

#266

Post by wmf »

KaiserPro wrote: post #206

I get where this is coming from, but no. We don't need to replace TCP in the datacentre. Why? because for things that are low latency, need rigid flow control, or other 99.99% utilisation case, one doesn't use TCP. (Storage, which is high throughput, low latency and has rigid flow control, doesn't [well ignore NFS and iscisi] use TCP) Look if it really was that much of a problem then everyone in datacentres would mov…

You're missing the fact that Stanford is the farm team for Google and Google is hyperscale. At scale, your "just spend more money" solutions are in fact more expensive than creating a new protocol. And like k8s, the new protocol can be sold to startups so they can "be like Google".

Re: We need a replacement for TCP in the datacenter [pdf]

#267

Post by ksec »

wmf wrote: post #266
KaiserPro wrote: post #206

I get where this is coming from, but no. We don't need to replace TCP in the datacentre. Why? because for things that are low latency, need rigid flow control, or other 99.99% utilisation case, one doesn't use TCP. (Storage, which is high throughput, low latency and has rigid flow control, doesn't [well ignore NFS and iscisi] use TCP) Look if it really was that much of a problem then everyone in datacentres would mov…

You're missing the fact that Stanford is the farm team for Google and Google is hyperscale. At scale, your "just spend more money" solutions are in fact more expensive than creating a new protocol. And like k8s, the new protocol can be sold to startups so they can "be like Google".

AWS is True HyperScale. Even more so than Google. And yet their spend more money solution on hardware seems to work fine.

Re: We need a replacement for TCP in the datacenter [pdf]

#268

Post by ElevenLathe »

cryptonector wrote: post #85

> Although Homa is not API-compatible with TCP, it should be possible to bring it into widespread usage by integrating it with RPC frameworks. Not being sockets API compatible kinda sucks. Ok, we could use a new connect() variation that allows for earlier data send, but the API being mostly similar would help -- there's a ton of socket code out there! As for RPC, well, RPC is mostly a thing of the past with most ever…

I don't think they mean RPC in the sense of rpc(3), but rather things like gRPC or protobuf. The point is that if both sides really just want to talk gRPC to each other (and never want to talk plain TCP to each other), then that use case is "easy" to meet by implementing gRPC on top of Homa.

Re: We need a replacement for TCP in the datacenter [pdf]

#269

Post by hinkley »

kortilla wrote: post #180
hinkley wrote: post #73

Earlier quoted context omitted.

> • In-order packet delivery This is a bit disingenuous, since it’s not the wire protocol but the kernel API that maintains the in order abstraction. With Jumbo packets you can still push a mountain of data without tripping up on “in order packet delivery” As developers we like this in order delivery to userspace because it vastly simplifies the code. We make up for the inefficiencies by processing dozens of hundreds…

Is it not considered a protocol violation to deliver out of order segments to the upper layer? That seems the same to me as abusing it to not require retransmits either. Remember, middle boxes can fully adhere to the TCP standards and terminate your TCP connection and enforce ordering. If you notice that, you’re not really following the protocol, you’re just using its header format.

Yes, but.

My read of the room is that he's conflating wire level and kernel level problems with userspace problems, which is a no-no because if Berkeley userspace has latency problems, we can deal with that separately from undoing 40 years of tribal knowledge in the process.

In the video he says that he was seeing 3x of theoretical latency to userspace that he fixed with Homa, but similar efforts to fix Berkeley Sockets saw 'almost a 2x' improvement which he deemed insufficient. A question I'd like to see answered over the next couple years is what IO APIs will be the most efficient in a world where io_uring is everywhere.

Re: We need a replacement for TCP in the datacenter [pdf]

#270

Post by hinkley »

pram wrote: post #120
hinkley wrote: post #91

Earlier quoted context omitted.

Spray and pray sounds like a reasonable fit for UDP, no? We’ve had these sorts of bottlenecks before, and they didn’t last. It’s always possible something fundamental changed, but it’s also possible that we are doing something wrong as the motherboard or OS levels and adopting new solutions puts us right back in that space where a couple of servers can easily saturate a network. If a network card can move data as fas…

The network is the computer™

Well, I mean yeah, that silly slogan is definitely rattling around in my head.
Post reply on HN