Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
NUMA Siloing in the FreeBSD Network Stack [pdf]
11–20 of 137 posts
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#12Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
> I get that, in part, this is a way to get people excited to work for Netflix It's even more amazing to me that this is a successful tactic in light of the fact that there's no shortage of people willing to work for ad-tech firms. If working for a company with good values was such a draw, you'd think it would give them more of a competitive advantage. Maybe the real reason is that Netflix's "good values" are genuine…
I don't see how netflix' "good values" are so much better than adtech. They're not exactly saving the planet, feeding the starving or growing artificial organs. Adtech is about getting eyeballs and converting that into money. Netflix is about getting eyeballs for hours and people paying for that privilege. Contributing to OSS projects they use is sort of the "not eating babies" baseline, even oracle and google do it!
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#13Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
It depends on how exactly each is tuned. Cache and buffer sizes, drivers, TCP congestion control algorithms, and more can affect this hugely, and are different. The defaults on linux (and probably on freebsd) also change reasonably often, so you can expect linux 4.x to have singificant differences from 5.x.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#14I assumed these were cache servers that went into ISPs, does anyone know what they mean by "Increases AWS cloud management overhead" on a couple of the slides?
Guessing that they have managed services in aws for maintaining all the nodes and where they are. If each box has 2x the ips, they would need more mgmt resources to keep track of what is on that box, if it's online, stats, etc.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#15Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
Realistically, Disney, or whomever competition, is not going to have enough technical expertise to deploy FreeBSD.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#16Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
If it's just serving files you don't necessarily need DPDK/XDP. For server-grade hardware there now is P2P-DMA and TLS accelerators which can offload everything to peripherials while still using normal socket APIs. You get NVMe -(PCIe)-> crypto accelerator -(PCIe)-> ethernet for the bulk of the data.
Neither CPU nor main memory see any of the network packets as long as they stay on the happy path. Only connection setup, DMA orchestration and occasional TLS renegotiation have to be handled.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#17Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
They also have their own CDN, to reduce load on ISPs.
https://media.netflix.com/en/company-blog/how-netflix-works-...
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#18Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
Agreed. I believe technology as a whole is light-years ahead of where it would be if companies treated all of their findings and system designs as closely-guarded secrets. I am relatively early in my career and I get to open-source a project that I have been working on for the first time and it feels great.
Just imagine how far ahead the other industries would be if we handled patent and trademark law like we do in the software industry.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#19Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…
I am actually not too sure whether it matters. The amount of people that have the skills to implement such a solution is not that high. There are are only so many people who can do high performance kernel, encryption and networking work.
They might give the advantages away if they release everything, so that others can simply deploy it or offer it as a hosted solution. But the pure documentation what they did are more like advertisements for interesting engineer work @ Netflix.
Re: NUMA Siloing in the FreeBSD Network Stack [pdf]
#20Is it still accurate that for network bound servers that FreeBSD still outperforms Linux?
I think once you get to a certain traffic level you are forced to do kernel-bypass stuff like DPDK. Regardless of Linux or FreeBSD being the kernel.
Netflix is not bypassing the FreeBSD kernel. They bypass userland for bulk data though --- with sendfile, and kTLS, their webserver (nginx) has to negotiate TLS and parse requests and open files, but then it's pretty simple.