Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a replacement for TCP in the datacenter [pdf]

web.stanford.edu

Post reply on HN
Print view View original

111–120 of 329 posts

Re: We need a replacement for TCP in the datacenter [pdf]

#111

Post by Joel_Mckay »

Dylan16807 wrote: post #103
Joel_Mckay wrote: post #99

Earlier quoted context omitted.

And "not having a standard response" is the antithesis of global communication. I appreciate your perspective though. Have a wonderful day.

I usually find that standard responses impede real communication.

The ISO OSI stratifies communication into layers, with lower layers tasked with the role of encapsulating meaningful human content for delivery across the networks. The protocol itself is unaware of its payload content in most circumstances, and this is why most network hardware has minimal complexity yet remains interoperable.

I assure you every packet of content is encapsulated in several standard responses.

Have a nice day =)

Re: We need a replacement for TCP in the datacenter [pdf]

#112

Post by duped »

throwaway892238 wrote: post #37

Yes!!! I have been saying for years that lower level protocols are a bad joke at this point, but nobody in the industry wants to invest in making things better. There are so many improvements we could be making, but corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. What's kind of hilarious about this paper is, these are just the network-layer probl…

Every major corporation has multiple research organizations doing nothing but invest in things that don't have immediate shareholder value.

What you're talking about though isn't just coming up with new ideas or even new products. It's replacing hundreds of billions in infrastructure wholesale. The scale at which these changes needs to happen to be practical are at the cluster level in a single data center. If you can propose something that fits that bill there are a few companies willing to pay you millions in salary as an engineering fellow to do it.

Re: We need a replacement for TCP in the datacenter [pdf]

#113

Post by cryptonector »

intelVISA wrote: post #32
wmf wrote: post #28

Earlier quoted context omitted.

You can use SCTP in a datacenter. Ousterhout el al. are surely aware of SCTP so I assume Homa is better in some way.

Forgive my ignorance but why isn't SCTP more frequently used in DCs? I know it misbehaves with home routers etc. but shouldn't be a factor here.

Software -- legacy software, which is always all software currently in use, which is an enormous code base.

It would be easier to have a drop-in replacement for TCP that, whenever it can work, connect() will use it, and which listen()/accept() will work with as well as TCP. Then all apps that can use TCP could use the new transport transparently.

Basically, we need a TCP++ that works with existing APIs but which can also provide new functionality via new APIs.

Of course, backwards-compatibility is very limiting, which sucks.

We can also have new transports that have new APIs, but we need a better TCP for backwards compatibility because legacy is forever.

Also, the focus on RPC is cool because any protocol where you typically have a library doing the I/O -and not too many such libraries- is amenable to using the new thing, and that includes HTTP (which isn't an RPC). But TFA really needs to mention HTTP in the same breath as RPC, because -sadly- way too many readers will just close the tab as soon as they see "RPC" and not "HTTP".

Re: We need a replacement for TCP in the datacenter [pdf]

#116

Post by scantydolt »

kebman wrote: post #110
oconnor663 wrote: post #87

Earlier quoted context omitted.

> 50 years of clock time and thousands of years of engineer time It's not just the size of the investment, it's that it's the protocol everyone uses to talk to other people's machines, and you can't upgrade or replace other people's machines.

Yes you can. Just offer a better product, and people will buy it instead of the old or bad product. Better yet, make the new product backwards compatible, and fewer people will have qualms about forking out for it. Better yet, do an aggressive takeover, like Microsoft did, and just force the entire industry to adopt your stuff...

Great! When do you think you'll have it done?

Re: We need a replacement for TCP in the datacenter [pdf]

#117

Post by kccqzy »

vlovich123 wrote: post #92
wmf wrote: post #76

Earlier quoted context omitted.

Google already does that.

I’m not aware of them using something other than TCP internally (I’m sure by now they’ve migrated to QUIC but I’m not sure that QUIC necessarily solves some of the scaling challenges / optimizes for gRPC and low latency).

I don't think Google-internal communications happen over gRPC. Maybe the protocol was design with an ambition to replace their internal RPC system but it probably failed at that.

They have a new system called Snap although judging from the paper I don't think it can completely replace TCP: https://research.google/pubs/pub48630/ My understanding is that Snap enables new use cases including moving functionality previously done via RPCs to RDMA-like one-sided operations. I think it is complement to RPCs but does not replace it.

Re: We need a replacement for TCP in the datacenter [pdf]

#118

Post by fomine3 »

xfs wrote: post #83
OmarAssadi wrote: post #75

Earlier quoted context omitted.

In addition to QUIC, KCP [1] is another reliable low-latency protocol that sits on top of UDP that might be interesting. And unlike RFC 9000/9001 (QUIC), encryption is optional. I haven't really seen it mentioned much outside of primarily China-focused projects, like V2Ray [2], but there is also some English information in their Git repo [2]. [1]: https://github.com/skywind3000/kcp > [2]: https://www.v2fly.org/en_US/…

KCP uses a brute force congestion control algorithm that is unfair and inefficient. It is also poorly specified, which is probably why it is less commonly used outside circumvention circles.

Still is looks interesting for some use cases, even though it's not fair if it's fully utilized on the internet.

Re: We need a replacement for TCP in the datacenter [pdf]

#120

Post by pram »

hinkley wrote: post #91
wmf wrote: post #79

Earlier quoted context omitted.

Ultimately one server cannot inject more than one link worth of traffic (e.g. 100 Gbps) into the network which is a tiny fraction of total capacity. Researchers have gotten really good results with "spray and pray" for sub-RTT flows combined with latency and queue depth feedback for multi-RTT flows.

Spray and pray sounds like a reasonable fit for UDP, no? We’ve had these sorts of bottlenecks before, and they didn’t last. It’s always possible something fundamental changed, but it’s also possible that we are doing something wrong as the motherboard or OS levels and adopting new solutions puts us right back in that space where a couple of servers can easily saturate a network. If a network card can move data as fas…

The network is the computer™
Post reply on HN