Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
NUMA Siloing in the FreeBSD Network Stack [pdf]
21–30 of 137 posts
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#22I assumed these were cache servers that went into ISPs, does anyone know what they mean by "Increases AWS cloud management overhead" on a couple of the slides?
Guessing that they have managed services in aws for maintaining all the nodes and where they are. If each box has 2x the ips, they would need more mgmt resources to keep track of what is on that box, if it's online, stats, etc.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#23Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#24Earlier quoted context omitted.
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
Author here.. We're doing ~200Gb/s through the kernel.
Thank you for your contributions. Great information.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#25Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
https://people.freebsd.org/~gallatin/talks/euro2019-ktls.pdf
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#26Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
Realistically, Disney, or whomever competition, is not going to have enough technical expertise to deploy FreeBSD.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#27Earlier quoted context omitted.
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
If it's just serving files you don't necessarily need DPDK/XDP. For server-grade hardware there now is P2P-DMA and TLS accelerators which can offload everything to peripherials while still using normal socket APIs. You get NVMe -(PCIe)-> crypto accelerator -(PCIe)-> ethernet for the bulk of the data. Neither CPU nor main memory see any of the network packets as long as they stay on the happy path. Only connection set…
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#28Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#29Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
Realistically, Disney, or whomever competition, is not going to have enough technical expertise to deploy FreeBSD.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#30Earlier quoted context omitted.
If it's just serving files you don't necessarily need DPDK/XDP. For server-grade hardware there now is P2P-DMA and TLS accelerators which can offload everything to peripherials while still using normal socket APIs. You get NVMe -(PCIe)-> crypto accelerator -(PCIe)-> ethernet for the bulk of the data. Neither CPU nor main memory see any of the network packets as long as they stay on the happy path. Only connection set…
In the work we've done, the TLS crypto of bulk data is handled entirely in the CPU by the kernel via ktls. Please see the slides, specifically the data flow diagrams.