Live data from Hacker News

We need a replacement for TCP in the datacenter [pdf]

web.stanford.edu

181–190 of 329 posts

Re: We need a replacement for TCP in the datacenter [pdf]

#181
post #7

It's interesting that everyone (including the author) talks about UDP as a lossy protocol, but it doesn't seem that UDP drops actually occur on a routine basis anywhere. The UDP-based DDOS attacks seem to prove that; if UDP really was being dropped, those DDOS attacks wouldn't be so problematic. That said, it's an interesting read. TCP is inefficient, but that inefficiency has been patched/masked by hardware solution…

> It's interesting that everyone (including the author) talks about UDP as a lossy protocol, but it doesn't seem that UDP drops actually occur on a routine basis anywhere.

Just to clarify, you are referring to in the datacenter right? They occur in wireless all of the time.

Re: We need a replacement for TCP in the datacenter [pdf]

#182

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

> It completely ignores that the "port number" abstraction for service identification has completely failed due to the industry glomming onto HTTP

If you have ever used multiple TCP or UDP connections in parallel on a single machine (doesn't matter if server or client) then you should realize that ports are required.

Apart from that, you can run HTTP on other ports than 80. You can also use HTTP to load balance or do service discovery by means of redirects. (Caveat, I don't work in this field and can't say how solid the approach works in practice).

Re: We need a replacement for TCP in the datacenter [pdf]

#183

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

> corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. This is ridiculous. Hyperscalars see an immediate ROI from efficiency/reliability improvements and actively invest in TCP alternatives all of the time. It's just really hard. Networking companies see an ability to differentiate their products from their peers and work on this kind of thing as well…

> FWIW, I wouldn't be surprised if the high-performance RDMA networks being put together for AI workloads were the thing that grew into the "next" thing.

Maybe we were just early in giving (HFT) customers RDMA back in ~2007[1][2] but I don't see it entering the mainstream anytime soon. And after a relatively short 20 years of adoption, the "next" thing for hyperscalers is not going to be the next thing for everyone else.

[1] https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2...

[2] https://www.thetradenews.com/wombat-and-voltaire-break-milli...

Re: We need a replacement for TCP in the datacenter [pdf]

#184
I am not really sure what Ousterhout means when talking "datacenter" but one of the key aspects of TCP networked applications is that it really doesn't matter if they are located in a datacenter, at the edge, in a mobile device, in a Raspberry at home or in your car and, better, they can be more or less moved from one hosting to another. Will this mean that application developers need to work with different network stacks at the same time?

Re: We need a replacement for TCP in the datacenter [pdf]

#185
post #170

Earlier quoted context omitted.

> It completely ignores that the "port number" abstraction for service identification has completely failed due to the industry glomming onto HTTP as some sort of universal tunnel encapsulation for all application-layer protocols I think this is more of an artefact of horizontal scaling and port-contention. De-facto standard discovery mechanism DNS does not work with ports, so "well-known port" abstraction kinda fail…

> De-facto standard discovery mechanism DNS does not work with ports Yes, it does, see SRV records.

I meant DNS A/AAAA queries with preconfigured/well-known ports being the default. While some applications/protocols/services do use some port discovery mechanism, I would argue it is nowhere close to being de-facto standard.

Re: We need a replacement for TCP in the datacenter [pdf]

#187
post #91
post #79

Earlier quoted context omitted.

Ultimately one server cannot inject more than one link worth of traffic (e.g. 100 Gbps) into the network which is a tiny fraction of total capacity. Researchers have gotten really good results with "spray and pray" for sub-RTT flows combined with latency and queue depth feedback for multi-RTT flows.

Spray and pray sounds like a reasonable fit for UDP, no? We’ve had these sorts of bottlenecks before, and they didn’t last. It’s always possible something fundamental changed, but it’s also possible that we are doing something wrong as the motherboard or OS levels and adopting new solutions puts us right back in that space where a couple of servers can easily saturate a network. If a network card can move data as fas…

You just described the motivation behind infiniband (and RDMA in general)

Re: We need a replacement for TCP in the datacenter [pdf]

#188
post #173

> The data model for TCP is a stream of bytes. However, this is not the right data model for most datacenter applications. Datacenter applications typically exchange discrete messages to implement remote procedure calls This isn't just a datacenter problem. Every single network protocol I've ever created or implemented is message based, not stream based. Every messaging system. Every video game. Every RPC transport.…

Because most protocols can handle message loss, with retransmit and proper ordering ? And we haven't started to talk about congestion yet… TCP is useful, and while I'd like to see a message get rid of one (or more) of those constraints to go with a a custom protocol, I feel like they'd be re-implementing the features in the end because these are very useful properties to have… Edit : the proposal in the article is ac…

No post body was provided.

Re: We need a replacement for TCP in the datacenter [pdf]

#189

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

> Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. If this was true then how do you explain that the likes of AWS, the same company w…

It's not obvious to me that replacing TCP really is harder than designing your "own" chip. Scarequotes here because those graviton chips (that's what you're referring to, I think?) are of course ARM chips, so they're not designing something fresh; they're adapting a very mature design to their own needs. In terms of interoperability, a custom chip based on a standard design is probably a simpler, more locally addressable problem than new network protocols.

Isn't it plausible that graviton was designed yet TCP retained simply because graviton as a project is easier to complete successfully?

Re: We need a replacement for TCP in the datacenter [pdf]

#190

Earlier quoted context omitted.

What is QUIC in your book? Given, say, $50 million of dev time, what would you go about fixing? And in what way?

Doesn't QUIC still run over TCP? I thought it was a replacement for HTTP not TCP (Edit: looks like it replaces TCP and HTTP)

I guess you were thinking about another clever name protocol, SPDY :-)

SPDY → HTTP/2

QUIC → HTTP/3

Post reply on HN