11–20 of 50 posts
Earlier quoted context omitted.
How did you get to six? Is that just sending a single packet without dealing with responses? Never thought about this before so I'm curious.
Source container -> VM -> host -> host -> VM -> destination container
For many years, the most popular routing platforms (i.e., boxes built by Cisco) performed IP packet forwarding and management functions on the same processor (often a RISC architecture). In cases where packet rate was high, it was possible for devices to become unresponsive or lose critical protocols responsible for sharing routing information. In the last 15 years there has been a hard move away from these architect…
Earlier quoted context omitted.
How did you get to six? Is that just sending a single packet without dealing with responses? Never thought about this before so I'm curious.
Source container -> VM -> host -> host -> VM -> destination container
This is one of the reasons I have a strong aversion to "cloud" technologies like docker and kubernetes. You take networking, something with decades of development and hardware support, and you put it all in the CPU. To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router. And so we develop things like DPDK to spend even more CPU ju…
> That's dumb. Is it? It's certainly inefficient compared to dedicated hardware. But so is anything relying on a CPU - we could just use ASICs for everything. But then every logical change requires weeks/months/years of development and manufacturing. The goal of k8s, VMs, etc is flexibility. I can set up a 100-node k8s cluster with less-than-perfectly-efficient networking stack in mere minutes. Good luck matching tha…
This is one of the reasons I have a strong aversion to "cloud" technologies like docker and kubernetes. You take networking, something with decades of development and hardware support, and you put it all in the CPU. To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router. And so we develop things like DPDK to spend even more CPU ju…
> That's dumb. Is it? It's certainly inefficient compared to dedicated hardware. But so is anything relying on a CPU - we could just use ASICs for everything. But then every logical change requires weeks/months/years of development and manufacturing. The goal of k8s, VMs, etc is flexibility. I can set up a 100-node k8s cluster with less-than-perfectly-efficient networking stack in mere minutes. Good luck matching tha…
Right, except the asics already exist and you're actively choosing the less efficient, more expensive option.
> The goal of k8s, VMs, etc is flexibility.
I don't think this is necessarily bad, as long as you understand the tradeoff. You're choosing a fundamentally slower architecture to make management easier. It's a choice of prioritizing the developer experience over the user experience.
Earlier quoted context omitted.
Source container -> VM -> host -> host -> VM -> destination container
It's not as bad as it sounds as a few of those transitions can be implemented via zero-copy APIs, so you're really just passing around pointers to shared memory.
This is one of the reasons I have a strong aversion to "cloud" technologies like docker and kubernetes. You take networking, something with decades of development and hardware support, and you put it all in the CPU. To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router. And so we develop things like DPDK to spend even more CPU ju…
This is one of the reasons I have a strong aversion to "cloud" technologies like docker and kubernetes. You take networking, something with decades of development and hardware support, and you put it all in the CPU. To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router. And so we develop things like DPDK to spend even more CPU ju…
I'm curious, what makes the natting/routing performance much better on a router? Is it that the router is an ASIC designed only to process packets vs a cpu that can execute any code?
Yes. Most routers + switches are "line rate" meaning the packets go through the switch at the speed of electricity, as if the switch wasn't there are all.
Earlier quoted context omitted.
> That's dumb. Is it? It's certainly inefficient compared to dedicated hardware. But so is anything relying on a CPU - we could just use ASICs for everything. But then every logical change requires weeks/months/years of development and manufacturing. The goal of k8s, VMs, etc is flexibility. I can set up a 100-node k8s cluster with less-than-perfectly-efficient networking stack in mere minutes. Good luck matching tha…
For vCPUs on the same physical multicore CPU, packets are sent by shared memory constructs which is very fast. It's effectively sending a pointer to the memory. I have seen transfers like this up to 80 Gbps, much faster than the physical NIC.
The transition from physical host memory -> vnic is analogous to just sending the packet over a wire. That's not the slow part of modern networks.