Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux network performance parameters

github.com

Post reply on HN
Print view View original

91–100 of 117 posts

Re: Linux network performance parameters

#91

Post by akoboldfrying »

pengaru wrote: post #84
superb_dev wrote: post #77

Earlier quoted context omitted.

Their criticism was accurate and well intentioned. Getting downvoted not for the content but perhaps poor phrasing is perfectly normal. Complaining at all about the votes your internet comment gets is asinine.

It's not asinine to complain that for no good reason a perfectly good technical reference written for the benefit of all readers was being grayed out (at the time) via downvotes. It took a non-zero amount of work to dig up where the setting is documented, and I didn't do it for my own benefit. This isn't taking it personally like I value HN karma. This is complaining purely because downvotes can make content invisibl…

>no good reason

Your original comment amounted to "It's working as documented, see here and here". But arguably the question was "Why does it work in this baffling way?"

Certainly that's how I interpreted it -- and while I didn't downvote your answer explaining that this weird footgun is actually documented behaviour, I got no value from either that information or your tone, which read to me as a little dismissive ("It's pretty clearly documented [, you lazy/incompetent person who didn't bother to look this up yourself]").

Re: Linux network performance parameters

#92

Post by cryptonector »

zamadatix wrote: post #89
cryptonector wrote: post #73

Nothing about PMTUD?

For TCP sockets I'd rather just MSS clamp on the internet gateway. On top of too many things just dropping PMTUD, enabling it results in a slower process while MSS clamping hijacks the initial TCP open messages directly.

There's PMTUD that doesn't depend on ICMP.

Re: Linux network performance parameters

#93

Post by raggi »

mikece wrote: post #4

[flagged]

This largely depends on how the application is written and in particular what if any non-POSIX interfaces it uses.

If you are looking to hit line rates with UDP, or looking to head well above ~1-10gbps with TCP, you're fast headed into territory where you likely need to move away from POSIX. (For a super dumb benchmark, oversized buffers amortizing syscall overhead might get you to 10gbps on TCP, but in a real application everything changes)

Once you're headed over 10gbps you'll quickly run into a hard need to retune things even for TCP, earlier if you're talking to non-local hosts. Once you're over 25gbps you're headed into the territory where you'll need to fix drivers, fix cpu tuning and so on. For a recent real world example: when we were doing performance analysis of our offloading patches for Tailscale we identified problems with the current default CPU frequency scaler for Intel CPUs on current kernels, and reached out to the maintainers with data.

Re: Linux network performance parameters

#96

Post by zamadatix »

cryptonector wrote: post #92
zamadatix wrote: post #89

Earlier quoted context omitted.

For TCP sockets I'd rather just MSS clamp on the internet gateway. On top of too many things just dropping PMTUD, enabling it results in a slower process while MSS clamping hijacks the initial TCP open messages directly.

There's PMTUD that doesn't depend on ICMP.

For TCP in Linux the only thing I know of is net.ipv4.tcp_mtu_probing=2 which is still slower than clamping at the edge. You can also run into weird slowdowns in cases with packet loss even after the initial discovery. If you don't have a way to clamp but absolutely need the interface to have jumbo enabled for local traffic performance it's probably the best fallback but even then I'm not sure it's worth the extra headache it causes.

Re: Linux network performance parameters

#97

Post by cperciva »

drewg123 wrote: post #87
heybrendan wrote: post #86

Earlier quoted context omitted.

You seem quite knowledgeable in this domain. Have you authored any blog posts to expand on this topic? I would welcome the chance to learn more from you.

I know a little bit about TCP, but I mainly focus on performance. I have several recent talks.. the slides are at: https://people.freebsd.org/~gallatin/talks/

Sure, you know a little bit about TCP, just like I know a little bit about EC2, and Paul Graham knows a little bit about startups.

Re: Linux network performance parameters

#98

Post by suprjami »

fragmede wrote: post #83
suprjami wrote: post #67

Earlier quoted context omitted.

> BBR Please stop. BBRv1 is broken and should not be used on the open internet. This sort of copy-paste cargo-cult performance tuning (just set a magical value and things will be better) is the exact opposite of what TFA is about. Thankfully Google are upstreaming BBRv3 so this will be over soon.

The original cargo cultists built runways on islands to cause supplies to be dropped off. It didn't work. If someone copy and pasted something they don't fully understand off the Internet, but it works , can you really blame them for it, or call them cargo cultists?

Except it doesn't work.

It's easy to coax BBR into converging on using 20% of a shared link instead of 50% (cohabiting with one other stream).

The inverse is true and it's easy to get BBR to hog 80% of a link instead of 50% (cohabiting with one other stream). If you're happy for other people to steal bandwidth from you with greedy CCAs then go ahead and ratelimit yourself. I'm not.

Re: Linux network performance parameters

#100

Post by fulafel »

drewg123 wrote: post #57
napkin wrote: post #50

Just changing Linux's default congestion control (net.ipv4.tcp_congestion_control) to 'bbr' can make a _huge_ difference in some scenarios, I guess over distances with sporadic packet loss and jitter, and encapsulation. Over the last year, I was troubleshooting issues with the following connection flow: client host reverse proxy host service host On average, I could not get better than 20% theoretical max throughput.…

The difference is that BBR does not use loss as a signal of congestion. Most TCP stacks will cut their send windows in half (or otherwise greatly reduce them) at the first sign of loss. So if you're on a lossy VPN, or sending a huge burst at 1Gb/s on a 10Mb/s VPN uplink, TCP will normally see loss, and back way off. BBR tries to find Bottleneck Bandwidth rate. Eg, the bandwidth of the narrowest or most congested link…

> Most TCP stacks will cut their send windows in half (or otherwise greatly reduce them) at the first sign of loss.

This was obsoleted by fast retransmit which was standardized in the 90s and ~everyone uses, right?

(loss generally is still used as a congestion signal, but first loss is usually not)

Post reply on HN