Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's time to replace TCP in the datacenter (2023)

arxiv.org

Post reply on HN
Print view View original

31–40 of 160 posts

Re: It's time to replace TCP in the datacenter (2023)

#31

Post by dataviz1000 »

bmitc wrote: post #24
tacitusarc wrote: post #16

Earlier quoted context omitted.

Websockets is a layer on top of TCP/IP.

Yes, I know that WebSockets layer over TCP/IP. But that both misses the point and is part of the point. The reason that I ask is that WebSockets seem to almost always be used in the context of web applications. TCP/IP still seems to dominate control communications between hardware. But why not WebSockets? Almost everyone ends up building a message framing protocol on top of TCP/IP, so why not just use WebSockets whic…

There isn't much of a difference between a router between two machines physically next to each other and a router in Kansas connecting a machine in California with a machine in Miami. The packets of data are wrapped with an address of where they are going in the header.

WebSockets are long lived socket connection designed specifically for use on the 'web'. TCP is data sent wrapped in packets that is ordered and guaranteed delivery. This causes a massive overhead cost. This is different from UDP which doesn't guarantee order and delivery. However, a packet sent over UDP might arrive tomorrow after it goes around the world a few times.

With fetch() or XMLHttpRequest, the client has to use energy and time to open a new HTTP connection while a WebSocket opens a long lived connection. When sending lots of bi directional messages it makes sense to have a WebSocket. However, a simple fetch() request is easier to develop. A developer needs to good reason to use the more complicated WebSocket.

Regardless, they both send messages using TCP which ensures the order of packets and guaranteed delivery which features have a lot to do with why TCP is the first choice.

There is UDP which is used by WebRTC which is good for information like voice or video which can have missing and unordered packets occasionally.

If two different processes on the same machine want to communicate, they can use a Unix socket. A Unix socket creates a special file (socket file) in the filesystem, which both processes can use to establish a connection and exchange data directly through the socket, not by reading and writing to the file itself. But the Unix Socket doesn't have to deal with routing data packets.

(ChatGPT says "Overall, you have a solid grasp of the concepts, and your statements are largely accurate with some minor clarifications needed.")

Re: It's time to replace TCP in the datacenter (2023)

#32

Post by wmf »

freetanga wrote: post #28

So, back to the mainframe and SNA in the data centers?

If Rosenblum can get an award for rediscovering mainframe virtualization, why not give Ousterhout an award for rediscovering SNA?

(SNA was before my time so I have no idea if Homa is similar or not.)

Re: It's time to replace TCP in the datacenter (2023)

#34

Post by Sebb767 »

UltraSane wrote: post #12

I wonder why Fibre Channel isn't used as a replacement for TCP in the datacenter. It is a very robust L3 protocol. It was designed to connect block storage devices to servers while making the OS think they are directly connected. OSs do NOT tolerate dropped data when reading and writing to block devices and so Fibre Channel has a extremely robust Token Bucket algorithm. The algo prevents congestion by allowing receiv…

> I wonder why Fibre Channel isn't used as a replacement for TCP in the datacenter

But it is often used for block storage in datacenters. Using it for anything else is going to be hard, as it is incompatible with TCP.

The problem with not using TCP is the same thing HOMA will face - anything already speaks TCP, nearly all potential hires know TCP and most problems you have with TCP have been solved by smart engineers already. Hardware is also easily available. Once you drop all those advantages, either your scale or your gains need to be massive to make that investment worth it, which is why TCP replacements are so rare outside of FAANG.

Re: It's time to replace TCP in the datacenter (2023)

#36

Post by detaro »

stiray wrote: post #20

How long did we need to support ipv6? Is it supported yet and more widely in use than the ipv4, like in mobile networks where everything is stashed behind NAT and ipv4 kept? Another protocol, something completely new? Good luck with that, i would rather bet on global warming to put us out of our misery (/s)... https://imgs.xkcd.com/comics/standards.png

Mobile networks especially are widely IPv6, with IPv4 being translated/tunneled where still needed. (End-user connections in general skew IPv6 in many places - it's observable how traffic patterns shift with people being at work vs at home. Corporate networks without IPv6 leading to more IPv4 traffic during the day, in the evening IPv6 from consumer connections takes over)

Re: It's time to replace TCP in the datacenter (2023)

#37

Post by parasubvert »

This has already been done at scale with HTTP/3 (QUIC), it's just not widely distributed beyond the largest sites & most popular web browsers. gRPC for example is still on multiplexed TCP via HTTP/2, which is "good enough" for many.

Though it doesn't really replace TCP, it's just that the predominant requirements have changed (as Ousterhout points out). Bruce Davie has a series of articles on this: https://systemsapproach.substack.com/p/quic-is-not-a-tcp-rep...

Also see Ivan Pepelnjak's commentary (he disagrees with Ousterhout): https://blog.ipspace.net/2023/01/data-center-tcp-replacement...

Re: It's time to replace TCP in the datacenter (2023)

#39

Post by YZF »

UltraSane wrote: post #12

I wonder why Fibre Channel isn't used as a replacement for TCP in the datacenter. It is a very robust L3 protocol. It was designed to connect block storage devices to servers while making the OS think they are directly connected. OSs do NOT tolerate dropped data when reading and writing to block devices and so Fibre Channel has a extremely robust Token Bucket algorithm. The algo prevents congestion by allowing receiv…

Are you suggesting some protocol layer of Fibre Channel to be used over IP over Ethernet?

TCP (in practice) runs on top of (mostly) routed IP networks and network architectures. E.g. a spine/leaf network with BGP. Fibre Channel as I understand it is mostly used in more or less point to point connections? I do see some mention of "Switched Fabric" but is that very common?

Re: It's time to replace TCP in the datacenter (2023)

#40

Post by t-writescode »

yesbut wrote: post #9

Another thing not worth investing time into for the rest of our careers. TCP will be around for decades to come.

True! And chances are, if you're developing website software or video game software, you'll never think about these sorts of things, it'll just be a dumb pipe for you, still.

And that's okay!

But there are other sorts of computer people than website writers and business application devs, and they're some of the people this would be interesting for!

Post reply on HN