Live data from Hacker News

NUMA Siloing in the FreeBSD Network Stack [pdf]

people.freebsd.org

71–80 of 137 posts

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#72
post #62

So why is it important to have a multisocket NUMA machine? Why not just save yourself a lot of hassle by having one socket? I know that the previous generation AMD machine had unavoidable NUMA but the new one doesn't.

It's money. Having a two-socket machine instead 2 one-socket machines, even when performance is only 80%, is saving both OPex and CAPex.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#73

Earlier quoted context omitted.

They also have their own CDN, to reduce load on ISPs. https://media.netflix.com/en/company-blog/how-netflix-works-...

you make it sound altruistic. it’s to avoid paying ISPs

It's to avoid paying last mile ISPs who have turned their monopolies into a mafia-like shakedown scheme.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#74
post #62

So why is it important to have a multisocket NUMA machine? Why not just save yourself a lot of hassle by having one socket? I know that the previous generation AMD machine had unavoidable NUMA but the new one doesn't.

One bigger host meams less management overhead. (see other comments about being u willing to run multiple ips and what not)

It may be a bit early to have a presentation from the latest Epyc processors. Most of the work was likely done with previous processors, but their slides said their AMD boxes are single socket.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#75
post #71

The Linux network stack has been the bane of my existence trying to squeeze more performance out of KeyDB. I really hope it gets this kind of love in the future. On Linux there’s a spinlock in do soft_irq that blocks even in non-blocking IO.

Have you tried optimising and benchmarking on a 2.6 kernel? I found the more recent kennels in rhel7 that the amount of kernel work being done very hard to control. The first core on any numa node is especially problematic and induce horrific jitter

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#76
post #62

So why is it important to have a multisocket NUMA machine? Why not just save yourself a lot of hassle by having one socket? I know that the previous generation AMD machine had unavoidable NUMA but the new one doesn't.

Density/power/cooling are real constraints when you aren't 100% bought-in to the cloud.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#77
post #62

So why is it important to have a multisocket NUMA machine? Why not just save yourself a lot of hassle by having one socket? I know that the previous generation AMD machine had unavoidable NUMA but the new one doesn't.

This talk is about Zen+ Epyc, not Zen2 (which is where the non-cache memory gets uniform). I don't know if they have release quality Epyc 7003 (Zen2) samples available yet, and if they do, NFLX probably isn't allowed to publish benchmarks about them. There's almost certainly still some value in their existing NUMA work even on Zen2, as things like L1/L2/L3 cache have locality even if memory and PCIe does not.

Pretty sure Intel single socket of this generation is totally non-viable for this workload due to lack of PCIe lanes. Maybe viable when Intel gets gen4 PCIe.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#78
post #69

Earlier quoted context omitted.

This is about TLS. DRM would probably be another layer of encryption inside?

TLS can be used as a DRM, preventing MITM caching attempts. However that also requires a client that cares

That isn't what's going on here.

Re: NUMA Siloing in the FreeBSD Network Stack [pdf]

#79

Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this? I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they h…

> ... just shares potential competitive advantages like this? Is it really a competitive advantage though? I would think that their brand and content library are the main ones. A CDN server doesn't really count IMHO: other streaming services can create their own appliances, though I'm guessing they'll use Linux due to its ubiquity. Though as a long-time FreeBSD fan and part-time user, I'm happy for the patches. I run…

> Is it really a competitive advantage though?

It totally is. These things get installed into ISP racks. ISPs have limited space and power for these things. One box pushing 200Gbps in the same space is twice as good (although, note they have 4x100Gbps NICs in this generation, there's room for more perf here).

Reducing the number of nodes helps with management of the network as well.

Post reply on HN