Live data from Hacker News

NUMA Siloing in the FreeBSD Network Stack [pdf]

people.freebsd.org

101–110 of 137 posts

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#101
post #61

Earlier quoted context omitted.

Not all of them are landed in CURRENT yet, much less stable/12. They'll be in 13.0; I can't speak to any future stable 12.2. On the other hand, CURRENT is pretty solid. A lot of folks, Netflix included, just run CURRENT.

Wait, they run CURRENT in production? Is that... safe?

I've been running CURRENT in production myself for about 2 years now. No problems so far.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#102
post #91
post #86

Earlier quoted context omitted.

TLS plays a different role. It makes sure you're receiving the Netflix's DRMed and encrypted content, and not some MITMed content that also happens to be correctly DRMed and encrypted, but contains something else.

Easy enough to make the content authenticate itself, instead of repeating cryptographic work for each viewer. E.g. every 64k block contains a SHA256 hash of the next 64k block. When a user seeks, you provide the hash of the first block they'll be receiving over a trusted channel. (It already can exist at start of stream RSA signed or whatever). But that's probably a pain when you have so many different devices to sup…

This exists in a more advanced form already

https://wicg.github.io/webpackage/draft-yasskin-http-origin-...

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#103

Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…

I like it, I think its spreading, and I think its a good thing for the world. "Top U.S. CEOs say companies should put social responsibility above profit" https://www.reuters.com/article/us-jp-morgan-business-roundt...

The part that worries me is that companies are increasingly seeing it as their social responsibility to gang up and destroy "bad" people like Stallman.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#104
post #70

Earlier quoted context omitted.

BSDs unwavering commitment to reliability. A Linux kernel crashes. A lot. FreeBSD ran cdrom.com for 365+ days without rebooting. Just saying

What's "a lot" here, and are you sure that's still true? Because my suspicion is that it might have been true before FAANG sunk billons of R&D into Linux, but I would be shocked if Linux kernels cash at any rare worth noticing today. Although it's splitting hairs, really; when is the last time you saw any major kernel crash?

I crash FreeBSD every day :)

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#105
post #62

So why is it important to have a multisocket NUMA machine? Why not just save yourself a lot of hassle by having one socket? I know that the previous generation AMD machine had unavoidable NUMA but the new one doesn't.

Communication over DDR4 is way faster than communication over PCIe Ethernet. Even 40Gbit is slow compared to RAM.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#106
post #79

Earlier quoted context omitted.

> ... just shares potential competitive advantages like this? Is it really a competitive advantage though? I would think that their brand and content library are the main ones. A CDN server doesn't really count IMHO: other streaming services can create their own appliances, though I'm guessing they'll use Linux due to its ubiquity. Though as a long-time FreeBSD fan and part-time user, I'm happy for the patches. I run…

> Is it really a competitive advantage though? It totally is. These things get installed into ISP racks. ISPs have limited space and power for these things. One box pushing 200Gbps in the same space is twice as good (although, note they have 4x100Gbps NICs in this generation, there's room for more perf here). Reducing the number of nodes helps with management of the network as well.

Yes, but the parent was talking sharing the competitive advantage of the patch.

But (a) what other streaming services even have CDN appliances, and (b) of the ones that do, which are using FreeBSD as a base?

If none of Netflix's competitors will use this patch, then it makes no material difference whether they share it or not.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#107
post #70

Earlier quoted context omitted.

BSDs unwavering commitment to reliability. A Linux kernel crashes. A lot. FreeBSD ran cdrom.com for 365+ days without rebooting. Just saying

What's "a lot" here, and are you sure that's still true? Because my suspicion is that it might have been true before FAANG sunk billons of R&D into Linux, but I would be shocked if Linux kernels cash at any rare worth noticing today. Although it's splitting hairs, really; when is the last time you saw any major kernel crash?

Every year I still see a Linux system or two live-lock. When I was still doing Solaris and FreeBSD admining that never happened.

However most systems aren't pushed to extremes, so these corner cases aren't seen very often, regardless of OS.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#108
post #71

The Linux network stack has been the bane of my existence trying to squeeze more performance out of KeyDB. I really hope it gets this kind of love in the future. On Linux there’s a spinlock in do soft_irq that blocks even in non-blocking IO.

> I really hope it gets this kind of love in the future.

Have you considered using FreeBSD? ;)

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#109
post #42

It’s my understanding that many organizations prefer *BSDs over Linux not because BSD is more performant ... but because you’re way more likely to get your organizations patch accepted into the upstream than Linux.

I doubt that is the case. The FreeBSD guys can be just as hardass about submissions as the Linux folks. However, there are considerations: 1) GPL vs BSD licenses. Companies like BSD licenses much more than GPL licenses. GPL adherents can whine all they want, but this is simply true. 2) BSD has a long history of having very good networking stacks--albeit on specific hardware. Linux supported everything initially--incl…

Re: 1 and 2.

Given that there are vendors that use FreeBSD for their appliances, they really don't want to have to send out techs to customers sites to fix things. So when the appliance makers choose hardware, they talk to component vendors about quality.

It's no surprise that you see commits from Intel and Chelsio employees in the FreeBSD logs: companies like Netflix, Isilon, NetApp, and Juniper partner with them to make sure things aren't buggy.

These collaborations lead to point 3.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#110

Earlier quoted context omitted.

From what I recall, the chelsio cards only support a mode of encryption suitable for storage devices, and it's not something you'd use for streaming media.

No, not true. The Chelsio card support GCM and CBC crypto in lookaside (like QAT) using the ccr(4) OCF driver, inline ("NIC TLS") with out - of - tree patches, and TLS offload in TOE mode. Now that ktls is upstream, we are looking at using the ccr crypto acceleration. We've already tested them in inline mode. TOE is not an option for us, since we do innovation in the TCP stack.

Thanks for clearing that up.
Post reply on HN