Various RDMA protocols were all about that. Where are they now?
It's time to replace TCP in the datacenter (2023)
111–120 of 160 posts
Re: It's time to replace TCP in the datacenter (2023)
#112Earlier quoted context omitted.
If anybody is gonna do it, it's gonna be someone like amazon that vertically integrates through most of the hardware
That’s my point: TCP in the datacenter remains a 1% problem, in the sense that only 1% of customers actually have this as a problem, and only 1% of those have the ability to invest in a solution. At that point, market conditions incentivize protecting their work and selling it to others (e.g., Public Cloud Service Providers) as opposed to releasing it into the wild as its own product line for general purchase (e.g.,…
Nevertheless Infiband exists.
Re: It's time to replace TCP in the datacenter (2023)
#113Earlier quoted context omitted.
But you might not need TCP. For example, using file-sockets between an app, db, and http server (rails+pgsql+nginx for example) has many benefits. The beauty of OSI layers.
Unix sockets can use tcp, udp, or be a raw stream https://en.wikipedia.org/wiki/Unix_domain_socket#:~:text=The... . Puma creates a `UnixServer` which is a ruby stdlib class, using the defaults, which is extending `UnixSocket` which is also using the defaults https://github.com/puma/puma/blob/fba741b91780224a1db1c45664... Those defaults are creating a socket of type `SOCK_STREAM`, which is a tcp socket > SOCK_STREAM w…
Hey! I know it’s muddled but that’s not quite correct. SOCK_STREAM is more general than TCP; SOCK_STREAM just means the socket is a byte stream. You would need to add IPPROTO_TCP on top of that to pull in the TCP stack.
UDS using SOCK_STREAM does not do that; ie, it is not using IPPROTO_TCP.
Re: It's time to replace TCP in the datacenter (2023)
#114This has already been done at scale with HTTP/3 (QUIC), it's just not widely distributed beyond the largest sites & most popular web browsers. gRPC for example is still on multiplexed TCP via HTTP/2, which is "good enough" for many. Though it doesn't really replace TCP, it's just that the predominant requirements have changed (as Ousterhout points out). Bruce Davie has a series of articles on this: https://systemsapp…
QUIC is not trying to solve the same problem as Ousterhout is. End user networks very different from datacenter.
Re: It's time to replace TCP in the datacenter (2023)
#115Various RDMA protocols were all about that. Where are they now?
Infiniband is still widely used in data centers, RDMA and IPoIB. Intel tried Omnipath but that died quickly (I don't know specifically why).
Re: It's time to replace TCP in the datacenter (2023)
#116Earlier quoted context omitted.
But you might not need TCP. For example, using file-sockets between an app, db, and http server (rails+pgsql+nginx for example) has many benefits. The beauty of OSI layers.
Unix sockets can use tcp, udp, or be a raw stream https://en.wikipedia.org/wiki/Unix_domain_socket#:~:text=The... . Puma creates a `UnixServer` which is a ruby stdlib class, using the defaults, which is extending `UnixSocket` which is also using the defaults https://github.com/puma/puma/blob/fba741b91780224a1db1c45664... Those defaults are creating a socket of type `SOCK_STREAM`, which is a tcp socket > SOCK_STREAM w…
Unix domain stream sockets do not use tcp. Nor do unix datagram sockets use udp. They're much simpler.
Re: It's time to replace TCP in the datacenter (2023)
#117Earlier quoted context omitted.
That’s my point: TCP in the datacenter remains a 1% problem, in the sense that only 1% of customers actually have this as a problem, and only 1% of those have the ability to invest in a solution. At that point, market conditions incentivize protecting their work and selling it to others (e.g., Public Cloud Service Providers) as opposed to releasing it into the wild as its own product line for general purchase (e.g.,…
Nevertheless Infiband exists.
Which makes the prospects of a replacement for either or both even more unlikely.
Re: It's time to replace TCP in the datacenter (2023)
#118Just using IP or UDP packets lets you implement something like Homa in userspace.
What's the advantage of making it a kernel level protocol?
Re: It's time to replace TCP in the datacenter (2023)
#119Earlier quoted context omitted.
Unix sockets can use tcp, udp, or be a raw stream https://en.wikipedia.org/wiki/Unix_domain_socket#:~:text=The... . Puma creates a `UnixServer` which is a ruby stdlib class, using the defaults, which is extending `UnixSocket` which is also using the defaults https://github.com/puma/puma/blob/fba741b91780224a1db1c45664... Those defaults are creating a socket of type `SOCK_STREAM`, which is a tcp socket > SOCK_STREAM w…
Unix domain stream sockets do not use tcp. Nor do unix datagram sockets use udp. They're much simpler.
>The type parameter should be one of two common socket types: stream or datagram.[10] A third socket type is available for experimental design: raw.
> SOCK_STREAM will create a stream socket. A stream socket provides a reliable, bidirectional, and connection-oriented communication channel between two processes. Data are carried using the Transmission Control Protocol (TCP).
> SOCK_DGRAM will create a datagram socket.[b] A Datagram socket does not guarantee reliability and is connectionless. As a result, the transmission is faster. Data are carried using the User Datagram Protocol (UDP).
> SOCK_RAW will create an Internet Protocol (IP) datagram socket. A Raw socket skips the TCP/UDP transport layer and sends the packets directly to the network layer.
I don't claim to be an expert, I just have a certain confidence that I'm able to comprehend words I read. It seems you can have 3 types of sockets, raw, udp, or tcp.
Re: It's time to replace TCP in the datacenter (2023)
#120Earlier quoted context omitted.
If anybody is gonna do it, it's gonna be someone like amazon that vertically integrates through most of the hardware
That’s my point: TCP in the datacenter remains a 1% problem, in the sense that only 1% of customers actually have this as a problem, and only 1% of those have the ability to invest in a solution. At that point, market conditions incentivize protecting their work and selling it to others (e.g., Public Cloud Service Providers) as opposed to releasing it into the wild as its own product line for general purchase (e.g.,…
Amazon's solutions to 1% problems are the next batch of cargo cult corporate solution speak that we're going to have to deal with. How can we have a web scale agile big data devops monorepo machine learning microservice oriented architecture if we're limited by TCP? I mean, it was developed in the 70s...