Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a replacement for TCP in the datacenter [pdf]

web.stanford.edu

Post reply on HN
Print view View original

31–40 of 329 posts

Re: We need a replacement for TCP in the datacenter [pdf]

#31

Post by jimmySixDOF »

jeffbee wrote: post #26

There's a little misleading point on page 2, or just a mistake. It says "Driving a 100 Gbps network at 80% utilization in both directions consumes 10–20 cores just in the networking stack" and it cites Google's Snap paper from 2019. But the Snap paper quite clearly says nothing like that. It says that Snap can drive a 100gbps NIC to 80gpbs with just 1.05 cores (Table 1) and that the whole-machine CPU load at 80gpbs i…

Yes that stat stood out for me too and I was wondering how to actually test this without breaking anything in the process.

Re: We need a replacement for TCP in the datacenter [pdf]

#32

Post by intelVISA »

wmf wrote: post #28
kevin_thibedeau wrote: post #6

If only we had a stream control transport protocol and were allowed to use it.

You can use SCTP in a datacenter. Ousterhout el al. are surely aware of SCTP so I assume Homa is better in some way.

Forgive my ignorance but why isn't SCTP more frequently used in DCs? I know it misbehaves with home routers etc. but shouldn't be a factor here.

Re: We need a replacement for TCP in the datacenter [pdf]

#33

Post by candiddevmike »

vore wrote: post #8
manv1 wrote: post #7

It's interesting that everyone (including the author) talks about UDP as a lossy protocol, but it doesn't seem that UDP drops actually occur on a routine basis anywhere. The UDP-based DDOS attacks seem to prove that; if UDP really was being dropped, those DDOS attacks wouldn't be so problematic. That said, it's an interesting read. TCP is inefficient, but that inefficiency has been patched/masked by hardware solution…

I think re UDP the point regarding it being unreliable is that you have to design your applications to take the unreliability into account, because it does happen even if it may be infrequent: assuming that it is reliable when you can get unreliable behavior will result in correctness issues.

At which point most applications end up reinventing a good chunk of TCP.

Re: We need a replacement for TCP in the datacenter [pdf]

#34

Post by colechristensen »

defrost wrote: post #5

Jumping to the end: > TCP is the wrong protocol for datacenter computing. > Every aspect of TCP’s design is wrong: there is no part worth keeping. I cannot disagree and Ousterhout argues well. > Homa offers an alternative that appears to solve all of TCP’s problems. I'm well behind the curve on protocols and now I have something to learn more about. > The best way to bring Homa into widespread usage is integrate it w…

> Every aspect of TCP’s design is wrong

The driver of most of a global network of computers which has been wildly successful beyond dreams before it was real… probably deserves a better deal than “every aspect is wrong”. It has worked fanatically well and chasing the long tail of performance improvements isn’t equivalent to determining what has gotten us here is wrong.

Re: We need a replacement for TCP in the datacenter [pdf]

#35

Post by [deleted] »

PixelOfDeath wrote: post #23
vore wrote: post #8

Earlier quoted context omitted.

I think re UDP the point regarding it being unreliable is that you have to design your applications to take the unreliability into account, because it does happen even if it may be infrequent: assuming that it is reliable when you can get unreliable behavior will result in correctness issues.

I see UDP more like a low level interface allowing you to build your own on top. Where you decide what packages need to be revived 100% and which one can be dropped. Basically the foundation of your very own TCP with hookers and blackjack.

[deleted]

Re: We need a replacement for TCP in the datacenter [pdf]

#36

Post by Spivak »

colechristensen wrote: post #34
defrost wrote: post #5

Jumping to the end: > TCP is the wrong protocol for datacenter computing. > Every aspect of TCP’s design is wrong: there is no part worth keeping. I cannot disagree and Ousterhout argues well. > Homa offers an alternative that appears to solve all of TCP’s problems. I'm well behind the curve on protocols and now I have something to learn more about. > The best way to bring Homa into widespread usage is integrate it w…

> Every aspect of TCP’s design is wrong The driver of most of a global network of computers which has been wildly successful beyond dreams before it was real… probably deserves a better deal than “every aspect is wrong”. It has worked fanatically well and chasing the long tail of performance improvements isn’t equivalent to determining what has gotten us here is wrong.

Everything about the protocol being wrong for the specific case of machines directly wired to one another over a high speed reliable network is not an admonishment of the protocol in general. And the protocol, being an abstract concept, doesn’t have feeling to hurt.

Re: We need a replacement for TCP in the datacenter [pdf]

#37

Post by throwaway892238 »

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo.

What's kind of hilarious about this paper is, these are just the network-layer problems! It completely ignores that the "port number" abstraction for service identification has completely failed due to the industry glomming onto HTTP as some sort of universal tunnel encapsulation for all application-layer protocols. And then there's all the non-backend problems!

And that's just TCP. We still lack any way to communicate up and down the stack of an entire transaction, for example for debugging purposes. We should have a way to forward every single layer of the stack across each hop, and return back each layer of the stack, so that we can programmatically determine the exact causes of network issues, automatically diagnose them, and inform the user how to solve them. But right now, you need a human being to jump onto the user's computer and fire up an assortment of random tools in mystical combinations and use human intuition to divine what's going on, like a god damn Networking Gandalf. And we've been doing it this way for 40+ years.

Re: We need a replacement for TCP in the datacenter [pdf]

#38

Post by Dylan16807 »

DonHopkins wrote: post #22

I'd tell you a UDP joke, but you probably wouldn't get it. So here's a TCP joke: Hello, would you like to hear a TCP joke? Yes, I'd like to hear a TCP joke. OK, I'll tell you a TCP joke. OK, I'll hear a TCP joke. Are you ready to hear a TCP joke? Yes, I am ready to hear a TCP joke. OK, I'm about to send the TCP joke. It will last 10 seconds, it has two characters, it does not have a setting, it ends with a punchline.…

A classic, but it's really far from how TCP actually works.

Re: We need a replacement for TCP in the datacenter [pdf]

#39

Post by 29athrowaway »

DonHopkins wrote: post #22

I'd tell you a UDP joke, but you probably wouldn't get it. So here's a TCP joke: Hello, would you like to hear a TCP joke? Yes, I'd like to hear a TCP joke. OK, I'll tell you a TCP joke. OK, I'll hear a TCP joke. Are you ready to hear a TCP joke? Yes, I am ready to hear a TCP joke. OK, I'm about to send the TCP joke. It will last 10 seconds, it has two characters, it does not have a setting, it ends with a punchline.…

The handshake sequence is exaggerated. It's just usually 3 messages.

The 3 initial messages establish a connection.

    A: I would like to tell you something. (SYN)

    B: I acknowledge you want to tell me something. (SYN-ACK)

    A: I received your acknowledgement. (ACK)
After the handshake sequence is done, data transfer begins.

Only then, it is possible to know that the "something" was a joke.

If B answers with RST instead of SYN-ACK the connection is refused. If B doesn't answer, A will interpret this as a connection timeout.

Post reply on HN