151–160 of 160 posts
Earlier quoted context omitted.
But you might not need TCP. For example, using file-sockets between an app, db, and http server (rails+pgsql+nginx for example) has many benefits. The beauty of OSI layers.
How do you think 'file-sockets' are implemented over a network?
> If Homa becomes widely deployed, I hypothesize that core congestion will cease to exist as a significant networking problem, as long as the core is not systemically overloaded. Yep. Sure; but, what happens when it becomes overloaded? > Homa manages congestion from the receiver, not the sender. [...] but the remaining scheduled packets may only be sent in response to grants from the receiver I hypothesize it will no…
RPC is something of a red flag as well. RPCs will never behave like local procedure calls, so the abstraction will always leak (the pendulum of popularity keeps swinging back and forth between RPC and special purpose protocols every few years, though).
Dumb question: why was it decided to only provide an unreliable datagram protocol in standard IP transit?
Because when you're sending a signal down a wire or through the air, fundamentally the communication medium only provides "Send it, maybe it arrives" At any time, the receiver could lose power. Or a burst of interference could disrupt the radio link. Or a backhoe could slice through the cable. Or many other things. IP merely reflects this physical reality.
Earlier quoted context omitted.
FC was much more expensive than ethernet, so needed a reason to be used. For block storage it is great, if slower than ethernet.
Is their a fundamental reason for it being more expensive than Ethernet or is just greed?
FC was one of the first non-mainframe specific storage area network fabrics. One of the key differences between FC and ethernet is the collision detection/avoidance and guaranteed throughput. All that extra coordination takes effort on big switches, so it cost more to develop/produce.
You could shovel data through it at "line speed" and know that it'll get there, or you'll get an error at the fabric level. Ethernet happily takes packets, and if it can't deliver, then it'll just drop them. (well, kinda, not all ethernet does that, because ethernet is the english language of layer 2 protocols)
Earlier quoted context omitted.
How do you think 'file-sockets' are implemented over a network?
They don’t have to use TCP. The point was to use sockets as the abstraction layer and use another inter-connect instead of TCP/IP. That way you’ve easily replaced TCP in the datacenter without major changes to many applications
This has already been done at scale with HTTP/3 (QUIC), it's just not widely distributed beyond the largest sites & most popular web browsers. gRPC for example is still on multiplexed TCP via HTTP/2, which is "good enough" for many. Though it doesn't really replace TCP, it's just that the predominant requirements have changed (as Ousterhout points out). Bruce Davie has a series of articles on this: https://systemsapp…
QUIC and Homa are not remotely similar and have completely different design constraints. I have no idea why people keep bringing up QUIC in this thread other than "It's another thing that isn't TCP." Yes, many things are not-TCP. The details are what matter.
QUIC is more focused on the global web applications, but most datacentres also leverage the web protocols (REST on HTTP 1.1 or HTTP/2, gRPC HTTP/2) for their inter-process communication, just with a a lot more east-west traffic (arguably 10x for every 1x N/S flow). There's also a fair amount app-specific messaging stacks (usually L7 over TCP) like Kafka, NATS or AMQP which have their own L7 facilities for dealing with TCP drawbacks that might benefit from a retrofit like Homa, but it's not clear if it's worth the effort.
They are design approaches for solving similar requirements. Yes, homa deals with other things (makes ECMP load balancing easier) but also has blindspots on datacenter requirements like security: a lot of data centre traffic requires hop by hop TLS for authentication, integrity and privacy, QUIC explicitly focuses on improving latency of TLS handshakes.
This has already been done at scale with HTTP/3 (QUIC), it's just not widely distributed beyond the largest sites & most popular web browsers. gRPC for example is still on multiplexed TCP via HTTP/2, which is "good enough" for many. Though it doesn't really replace TCP, it's just that the predominant requirements have changed (as Ousterhout points out). Bruce Davie has a series of articles on this: https://systemsapp…
QUIC is not trying to solve the same problem as Ousterhout is. End user networks very different from datacenter.
> For many years, RDMA NICs could cache the state for only a few hundred connections; if the number of active connections exceeded the cache size, information had to be shuffled between host memory and the NIC, with a considerable loss in performance. A massively parallel task? Sounds like something doable with GPGPU.
Now the information has to be shuffled between the NIC and host memory and the GPU.
Earlier quoted context omitted.
Unix domain stream sockets do not use tcp. Nor do unix datagram sockets use udp. They're much simpler.
>The type parameter should be one of two common socket types: stream or datagram.[10] A third socket type is available for experimental design: raw. > SOCK_STREAM will create a stream socket. A stream socket provides a reliable, bidirectional, and connection-oriented communication channel between two processes. Data are carried using the Transmission Control Protocol (TCP). > SOCK_DGRAM will create a datagram socket.…
Earlier quoted context omitted.
> Even if they do (e.g., Google with QUIC), the broad vibe I get is that folks aren’t likely to trust those offerings as lacking in ulterior motives. It's pretty unfortunate that we've landed here. Hordes of venture-backed companies building shareware-like software with an "open source" label has done some severe damage.
Which is ironic, because I remember TCP/IP maturing in the protocol wars of the 90s. My Cisco course specifically covered the protocols separately from the media layers because you couldn’t know if your future employer still leveraged Token Ring, or ATM, or IPX, or TCP; a decade later, the course had drastically simplified to “ethernet” and “TCP/IP” only. Many of these came from companies who created the protocol sol…
Good times. But it didn't matter because ATM was the future. /s
Many of these came from companies who created the protocol solely to push products
Like 100Base-VG? That was a good laugh.
TCP/IP won out for general use because of its low cost of deployment and ongoing support compared to alternatives, and that point is lost on the modern engineer that’s just looking at this stuff as “paper problems”
Welcome to my world...