Also wondering, why not just use DPDK?
I think they are getting a lot of value from the rest of the Kernel's networking (VETH/namespaces etc talking to containers).
An eBPF loophole: Using XDP for egress traffic
11–20 of 81 posts
Re: An eBPF loophole: Using XDP for egress traffic
#12Also wondering, why not just use DPDK?
First, "just use" is doing a lot of work in that sentence, because DPDK is much harder to use than XDP. The authors of this blog were surprised they had to do their own checksumming, for instance.
Maybe more importantly: they're not building a middlebox. DPDK ultra-high performance comes in part from polling. It's always running. XDP is just an extension to the existing network driver.
Re: An eBPF loophole: Using XDP for egress traffic
#13[flagged]
Re: An eBPF loophole: Using XDP for egress traffic
#14Hi HN, Shivansh (founder) here, happy to answer any questions folks might have about the implementation and the benchmarks!
I come from a very different world (optimizing the FreeBSD kernel for the Netflix CDN, running on bare metal) but performance leaps like this are fascinating to me.
One of the things that struck me when reading this with only general knowledge of the linux kernel is: What makes things so terrible? Is iptables really that bad? Is something serialized to a single core somewhere in the other 3 scenarios? Is the CPU at 100% in all cases? Is this TCP or UDP traffic? How many threads is iperf using? It would be cool to see the CPU utilization of all 4 scenarios, along with CPU flamegraphs.
Re: An eBPF loophole: Using XDP for egress traffic
#15Hi HN, Shivansh (founder) here, happy to answer any questions folks might have about the implementation and the benchmarks!
I come from a very different world (optimizing the FreeBSD kernel for the Netflix CDN, running on bare metal) but performance leaps like this are fascinating to me. One of the things that struck me when reading this with only general knowledge of the linux kernel is: What makes things so terrible? Is iptables really that bad? Is something serialized to a single core somewhere in the other 3 scenarios? Is the CPU at 1…
It's also a bit depressing that everyone is still using the slower iptables, when nftables has been in the kernel for over a decade.
Re: An eBPF loophole: Using XDP for egress traffic
#16For NAT (Network Address Translation) or any other packet header modifications, you need to recalculate checksums manually
Why doesn’t checksum offload in the NIC take care of that?
Re: An eBPF loophole: Using XDP for egress traffic
#17I understand they are attached to the phrase "loophole" but it feels fairly like they are using it as designed to me?
XDP is intended only for inbound traffic. They are exploiting veth pairs to make outbound traffic "look like" inbound traffic. That's the "loophole".
Re: An eBPF loophole: Using XDP for egress traffic
#18Really good, and glad that you're taking this technique further into a docker network plugin. I wouldn't be surprised to see a Kubernetes CNI appear using this approach, seems entirely viable unless I am missing something.
I'll definitely be coming to check you all out at Kubecon.
Re: An eBPF loophole: Using XDP for egress traffic
#19I think the title is a little disingenuous and the idea of using a redirect is certainly not novel. The solution for XDP egress should be able to handle all host egress including sr-iov traffic. This works with a very specific namespace driven topology.
Re: An eBPF loophole: Using XDP for egress traffic
#20XDP (eXpress Data Path) is the fastest packet processing framework in linux - but it only works for incoming (ingress) traffic. We discovered how to use it for outgoing (egress) traffic by exploiting a loophole in how the linux kernel determines packet direction. Our technique delivers 10x better performance than current solutions, works with existing Docker/Kubernetes containers, and requires zero kernel modificatio…
Forgive me of my ignorance, but is XDP faster than DPDK for packet processing? It seems like DPDK has had a lot of work done for hardware optimizations that allow speeds that I can’t recall XDP being able to do. I have not looked too deeply into this though, so I’m very open to being wrong!
DPDK is a framework with multiple backends, on the receive side it can use XDP to intercept packets.
You can't compare the efficiency of the frameworks without talking about the specific setups on the host. The major advantage of XDP is that it is completely baked into the kernel. All you need to do is bring your eBPF program and attach it. DPDK requires a great deal of setup and user space libraries to work.