Live data from Hacker News

Linux network performance parameters

github.com

11–20 of 117 posts

Re: Linux network performance parameters

#11
post #5
post #4

[flagged]

This is kind of an urban legend, do you think the multi millions servers from Google, Amazon etc... have those performance issues?

The big guys don't have the patience to wait for Linux kernel networking to be fast and scalable. They bypass the kernel and take over the hardware.

https://blog.acolyer.org/2019/11/11/snap-networking/

Re: Linux network performance parameters

#12
post #11
post #5

Earlier quoted context omitted.

This is kind of an urban legend, do you think the multi millions servers from Google, Amazon etc... have those performance issues?

The big guys don't have the patience to wait for Linux kernel networking to be fast and scalable. They bypass the kernel and take over the hardware. https://blog.acolyer.org/2019/11/11/snap-networking/

That's funny ... the "big guys" are some of the biggest contributors to the Linux network stack, almost as if they were actually using it and cared about how well it works.

Re: Linux network performance parameters

#13
post #12
post #11

Earlier quoted context omitted.

The big guys don't have the patience to wait for Linux kernel networking to be fast and scalable. They bypass the kernel and take over the hardware. https://blog.acolyer.org/2019/11/11/snap-networking/

That's funny ... the "big guys" are some of the biggest contributors to the Linux network stack, almost as if they were actually using it and cared about how well it works.

History has shown that tons of Linux networking scalability and performance contributions have been rejected by the gatekeepers/maintainers. The upstream kernel remains unsuitable for datacenter use, and all the major operators bypass or patch it.

Re: Linux network performance parameters

#14
post #5
post #4

[flagged]

This is kind of an urban legend, do you think the multi millions servers from Google, Amazon etc... have those performance issues?

This is "appeal to authority" fallacy incarnate.

Google/Amazon "etc;" are likely happy to pay the cost because it really is "good enough" and the benefits of Linux over FreeBSD are otherwise quite considerable.

Google in particular seems blissfully happy to literally throw hardware at problems; since hardware is (for them especially) fundamentally extremely cheap.

Even multiple percentage gains in throughput are not necessary for most applications, and Linux is decent enough with latency if you avoid having complex IP/NFTables rules and avoid CONNTRACK like the plague.

as u/jeffbee says anyway, most of the larger tech companies these days are using userland networking and bypass the kernel almost completely for networking.

Re: Linux network performance parameters

#15

Could anyone recommend a video or video series covering similar material? There lots on networking in general, but I've had a hard time finding some on Linux specific implementation

I'm also seconding this, but from microcontroller perspective.

I want to try developing a simple tcp echo server for a microcontroller, but most examples just use the vendor's own tcp library and put no effort explaining how to manually setup and establish connection to the router.

Re: Linux network performance parameters

#16
Does performance tuning for Wi-Fi adapters matter?

On desktops, other than disabling features, can anything fix the problems with i210 and i225 ethernet? Those seem to be the two most common NICs nowadays.

I don't really understand why common networking hardware and drivers are so flawed. There is a lot of attention paid to RISC-V. How about start with a fully open and correct NIC? They'll shove it in there if it's cheaper than an i210. Or maybe that's impossible.

Re: Linux network performance parameters

#17
post #14
post #5

Earlier quoted context omitted.

This is kind of an urban legend, do you think the multi millions servers from Google, Amazon etc... have those performance issues?

This is "appeal to authority" fallacy incarnate. Google/Amazon "etc;" are likely happy to pay the cost because it really is "good enough" and the benefits of Linux over FreeBSD are otherwise quite considerable. Google in particular seems blissfully happy to literally throw hardware at problems; since hardware is (for them especially) fundamentally extremely cheap. Even multiple percentage gains in throughput are not…

I know they bypass the kernel but my point still stands, most of the servers on the internet runs on Linux, that's a fact, so there was more money, time invested, man power on that OS than any others.

Re: Linux network performance parameters

#18
post #17
post #14

Earlier quoted context omitted.

This is "appeal to authority" fallacy incarnate. Google/Amazon "etc;" are likely happy to pay the cost because it really is "good enough" and the benefits of Linux over FreeBSD are otherwise quite considerable. Google in particular seems blissfully happy to literally throw hardware at problems; since hardware is (for them especially) fundamentally extremely cheap. Even multiple percentage gains in throughput are not…

I know they bypass the kernel but my point still stands, most of the servers on the internet runs on Linux, that's a fact, so there was more money, time invested, man power on that OS than any others.

Your point is that popularity means that it will improve.

This is true, to a point.

Counterpoint: Windows Desktop Experience.

EDIT: that comment was glib, let me do a proper counterpoint.

Common area's are some of the most least maintained in reality; I can think of meet-me-rooms or central fibre hubs in major cities; they are expensive and subject to a lot of the whims of the major provider.

Crucially, despite large amounts of investment, the underlying architecture or infrastructure remains, even if the entire fabric of the area changes around it. Most providers using these kind of common areas do everything they can to avoid touching the area itself, especially as after a while it becomes very difficult to navigate and politically charged.

Fundamentally the architecture of Linux's network stack, really is, "good enough", which is almost worse than you would originally think since "good enough" means there's no reason to look there. There is an old parable about "worse is better" because if something is truly broken people will put effort into fixing it.

Linux's networking stack is fine, it's just not quite as good an architecture as the FreeBSD one. FreeBSD one has a lot less attention on it but fundamentally it's a cleaner implementation and easier to get much more out of..

You will find the same argument ad infinitum regarding other subjects such as Epoll vs IOCP vs kqueue (Epoll was abysmally terrible though and ended up being replaced by IO_URING, but even that took over a decade)

Re: Linux network performance parameters

#19
post #5
post #4

[flagged]

This is kind of an urban legend, do you think the multi millions servers from Google, Amazon etc... have those performance issues?

Do you think that (outside of a few special cases) they're using anything near the network bandwidth available to them?

I would expect in the 1% to 10% bandwidth utilization, on average. From my vague recollection, that's what it was at FB when I was there. They put stupid amounts of network capacity in so that the engineers rarely have to think about the capacity of the links they're using, and that if their needs grow, they're not bottlenecked on a build out.

To answer the original question, it's complicated. I have a weird client where freebsd gets 450 MiB/s, and Linux gets 85 with the default congestion control algorithm. Changing the congestion control algorithm can get me between 1.7 MiB/s and 470 MiB/s. So, better performance... Under what circumstances?

Re: Linux network performance parameters

#20
post #4

[flagged]

Performance parity on which axis? For which use case?

Talking generally about "network performance" is approximately as useful as talking generally about "engine performance". Just like it makes no sense to compare a weed-eater engine to a locomotive diesel without talking about use case and desired outcomes, it makes no sense to compare "performance of FreeBSD network stack" and "Linux network stack" without understanding the role those systems will be playing in the network.

Depending on context, FreeBSD, Linux or various userland stacks can be a great, average, or terrible choices.

Post reply on HN