Live data from Hacker News

We need a replacement for TCP in the datacenter [pdf]

web.stanford.edu

61–70 of 329 posts

Re: We need a replacement for TCP in the datacenter [pdf]

#61

At a glance, this sounds very similar to AWS' SRD protocol [1] - I'm curious how they compare but I see no mention of SRD in this paper. [1] https://assets.amazon.science/a6/34/41496f64421faafa1cbe301c...

Sounds very familiar to a lot of ideas that have been proposed as improvements to TCP over the years.

Re: We need a replacement for TCP in the datacenter [pdf]

#62
post #26

There's a little misleading point on page 2, or just a mistake. It says "Driving a 100 Gbps network at 80% utilization in both directions consumes 10–20 cores just in the networking stack" and it cites Google's Snap paper from 2019. But the Snap paper quite clearly says nothing like that. It says that Snap can drive a 100gbps NIC to 80gpbs with just 1.05 cores (Table 1) and that the whole-machine CPU load at 80gpbs i…

Yes that stat stood out for me too and I was wondering how to actually test this without breaking anything in the process.

> Yes that stat stood out for me too and I was wondering how to actually test this without breaking anything in the process.

dpdk has been doing just that for quite some time now. perhaps you can try that and see ?

Re: We need a replacement for TCP in the datacenter [pdf]

#63

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

Your ideas are interesting, can you link to or explain a concrete example though? The idea of everything magically debugging itself doesn't apply to a single piece of software I've ever seen, so I'm curious what kind of design would lead to that being possible.

Heres an example of an improvement to sending large files over long distances -- Tsunami protocol. It tries to get a best of both worlds to limit the detrimental effect of synchronous roundtrips in the TCP protocol for file transfers:

https://tsunami-udp.sourceforge.net/

Re: We need a replacement for TCP in the datacenter [pdf]

#64

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

What is QUIC in your book? Given, say, $50 million of dev time, what would you go about fixing? And in what way?

Doesn't QUIC still run over TCP? I thought it was a replacement for HTTP not TCP (Edit: looks like it replaces TCP and HTTP)

Re: We need a replacement for TCP in the datacenter [pdf]

#66

Earlier quoted context omitted.

The handshake sequence is exaggerated. It's just usually 3 messages. The 3 initial messages establish a connection. A: I would like to tell you something. (SYN) B: I acknowledge you want to tell me something. (SYN-ACK) A: I received your acknowledgement. (ACK) After the handshake sequence is done, data transfer begins. Only then, it is possible to know that the "something" was a joke. If B answers with RST instead of…

You're missing the frame size comms that then occur following this.

[deleted]

Re: We need a replacement for TCP in the datacenter [pdf]

#67
post #56

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

Surely most of your ideas are already being deployed in QUIC/HTTP3. It just happens inside a UDP datagram, for compatibility. Really you're not going to see any new IP protocol layers, there's too much quirky hardware on the network that wouldn't be able to handle it. If we can't even get IPv6 to work all the way to the client, we're never seeing new values for the protocol byte.

Don’t the hyperscaled cloud providers run totally segmented networks? What’s stopping them from using something proprietary internally and just exposing TCP at the end for termination of client connections?

Re: We need a replacement for TCP in the datacenter [pdf]

#68

Earlier quoted context omitted.

What is QUIC in your book? Given, say, $50 million of dev time, what would you go about fixing? And in what way?

Doesn't QUIC still run over TCP? I thought it was a replacement for HTTP not TCP (Edit: looks like it replaces TCP and HTTP)

Nope, UDP.

Re: We need a replacement for TCP in the datacenter [pdf]

#69

Earlier quoted context omitted.

What is QUIC in your book? Given, say, $50 million of dev time, what would you go about fixing? And in what way?

Doesn't QUIC still run over TCP? I thought it was a replacement for HTTP not TCP (Edit: looks like it replaces TCP and HTTP)

QUIC runs over UDP, and provides streams and encryption. HTTP/3 is designed to take advantage of QUIC streams (replacing HTTP/2 streams which were problematic due to TCP head of line blocking).

The RFCs are a bit elaborate so folks interested might want to look at this instead[1], which has one of the RFC authors explaining the basics of QUIC and HTTP/3.

[1] https://www.youtube.com/watch?v=cdb7M37o9sU

Re: We need a replacement for TCP in the datacenter [pdf]

#70
post #20

Earlier quoted context omitted.

It's theoretically much easier to introduce a new transport inside of a DC, since you're inside the network perimeter and you'll generally have control over policy-based filtering decisions.

The context here is TFA advocating for use of higher-level message-oriented frameworks instead of raw socket APIs so that they can use a non-TCP transport without changing the application code.

Within a DC/cloud provider network. This isn’t about protocols you’d see traversing the public internet until the cloud providers see value in a protocol and start to push it out through IETF (eg see QUIC which was done by a company that owned both the browser and the data center). If there’s a “small” SW improvement that lets you use your HW 10x more efficiently that’s totally worth it given the end of scaling. You either invest in SW or pay for custom ASIC development. You’re not getting a free lunch anymore by just waiting a few years and getting that 10x gain for “free”.
Post reply on HN