I've always found Nagle's algorithm being a kernel-level default quite silly. It should be up to the application to decide when to send and when to buffer and defer.
It’s up to the application to change this parameter on per-socket basis
It's Always TCP_NODELAY
121–130 of 186 posts
Re: It's Always TCP_NODELAY
#122Earlier quoted context omitted.
At this point, this is an application level problem and not something the kernel should be silently doing for you IMO. An option for legacy systems or known problematic hosts fine, but off by default and probably not a per SOCKOPT. Every modern language has buffers in their stdlib. Anyone writing character at a time to the wire lazily or unintentionally should fix their application.
So to be clear, you believe every program that outputs a bulk stream to stdout should be written to check if stdout is a socket and enable Nagle's algorithm if so? That's not just busywork - it's also an abstraction violation. By explicitly turning off Nagle's, you specify that you understand TCP performance and don't need the abstraction, and this is a reasonable way to do things. Imagine if the kernel pinned thread…
No, the program should take care to enable TCP_NODELAY when creating the socket. If the program gets passed a FD from outside it's on the outside program to ensure this. If somehow the program very often gets outside FDs from an oblivious source that could be a TCP socket, then it might indeed have to manually check if it really wants Nagle's algorithm.
Re: It's Always TCP_NODELAY
#123The Nagle algorithm was created back in the day of multi-point networking. Multiple hosts were all tied to the same communications (Ethernet) channel, so they would use CSMA ( https://en.wikipedia.org/wiki/Carrier-sense_multiple_access_... ) to avoid collisions. CSMA is no longer necessary on Ethernet today because all modern connections are point-to-point with only two "hosts" per channel. (Each host can have any nu…
False. It really was just intended to coalesce packets. I’ll be nice and not attack the feature. But making that the default is one of the biggest mistakes in the history of networking (second only to TCP’s boneheaded congestion control that was designed imagining 56kbit links)
> (second only to TCP’s boneheaded congestion control that was designed imagining 56kbit links)
What would you change here?
Re: It's Always TCP_NODELAY
#124What we need is configurable ack packet counts.
Then we can make TCP become UDP.
And then we solved everything.
Both linux and Windows have this config but it's buggy so we're back to TCP and UDP.
Re: It's Always TCP_NODELAY
#125What we need is configurable ack packet counts. Then we can make TCP become UDP. And then we solved everything. Both linux and Windows have this config but it's buggy so we're back to TCP and UDP.
I’m curious about this. Can you share more details or some links that discuss what you’re describing?
Re: It's Always TCP_NODELAY
#126Earlier quoted context omitted.
Just to add, ethernet uses csma/cd, WiFi uses csma/ca. Upgraded our DC switches to new ones around 2014 and needed to keep a few old ones because the new ones didn't support 10Mbit half duplex.
What did you still need to connect with 10mbit half duplex in 2014? I had gigabit to the desktop for a relatively small company in 2007, by 2014 10mb was pretty dead unless you had something Really Interesting connected....
If you worked in an industrial setting, legacy tech abounds due to the capital costs of replacing the equipment it supports (includes manufacturing, older hospitals, power plants, and etc). Many of these even still use token ring, coax, etc.
One co-op job at a manufacturing plant I worked at ~20 years ago involved replacing the backend core networking equipment with more modern ethernet kit, but we had to setup media converters (in that case token ring to ethernet) as close as possible to the manufacturing equipment (so that token ring only ran between the equipment and the media converter for a few meters at most).
They were "lucky" in that:
1) the networking protocol that was supported by the manufacturing equipment was IPX/SPX, so at least that worked cleanly on ethernet and newer upstream control software running on an OS (HP-UX at the time)
2) there were no lives at stake (eg nuclear safety/hospital), so they had minimal regulatory issues.
Re: It's Always TCP_NODELAY
#127Earlier quoted context omitted.
the whole point of TCP is that it is a stream of bytes, not of messages. The problem is that this is not in practice quite what most applications need, but the Internet evolved towards UDP and TCP only. So you can have message-based if you want, but then you have to do sequencing, gap filling or flow control yourself, or you can have the overkill reliable byte stream with limited control or visibility at the applicat…
For me, the “whole point” of TCP is to add various delivery guarantees on top of IP. It does not mandate or require a particular API. Of course, you can provide a stream API over TCP which suits many applications but it does not suit all and by forcing this abstraction over TCP you end up making message oriented applications (e.g request /response type protocols) more complex to implement than if you had simply expos…
TCP is not message-oriented. Retransmitted bytes can be at arbitrary offsets and do not need to align with the way the original transmission was fragmented or even an earlier retransmission.
Re: It's Always TCP_NODELAY
#128Earlier quoted context omitted.
Indeed you can, but we've found it useful to use MSG_MORE when using state machines, where different states are responsible for different parts of the reply. (Plenty of examples in states*.c here: https://gitlab.com/nbdkit/libnbd/-/tree/master/generator?ref... )
Doing more system calls isn't really a good idea for performance. Also if you're doing asynchronous writes you typically can only have one write in-flight at any time, you should aggregate all other buffers while that happens. Though arguably asynchronous writes are often undesired due to the complexity of doing flow-control with them.
Actually, with newer Linux kernels and io_uring, it appears it is now possible to do multiple writes asynchronously concurrently, by annotating each one with a sequencing constraint.
Whether that's really useful or not depends on whether you do the associated buffer management work.
Re: It's Always TCP_NODELAY
#129I swear, it seems like I’ve seen some variation of this 50 times on HN in the past 15 years. The core issue with Nagle’s algorithm (TCP_NODELAY off) is its interaction with TCP Delayed ACK. Nagle prevents sending small packets if an ACK is outstanding, while the receiver delays that ACK to piggyback it on a response. When both are active, you get a 200ms "deadlock" where the sender waits for an ACK and the receiver w…
> Proper usage means disabling Nagle at the socket level but managing your own buffering in user-space.
Honestly, if you're writing a (cough) typical webservice that just serializes an object to JSON, you've pretty much done that. Nagle just slows that situation down, and TCP_NODELAY should always be enabled in that situation. (Granted, if you're using newer HTTP (3? SPDY?) you probably aren't even on TCP and don't even need to bother.)
It's only when sending large payloads that you might have to think about buffering.
Re: It's Always TCP_NODELAY
#130Why doesn’t linux just add a kconfig that enables TCP_NODELAY system wide? It could be enabled by default on modern distros.
Perhaps you can set up iptables rule to add the bit.