Brings back memories. This was a big Sybase performance win back in the day.
It's always TCP_NODELAY
231–240 of 276 posts
Re: It's always TCP_NODELAY
#232Earlier quoted context omitted.
Well, isn't that already how it works? If I physically unplug my ethernet cable, won't TCP-related syscalls start failing immediately?
Probably, but I don't know how the physical layers work underneath. But regardless, it's trivial to just monitor something constantly to ensure the connection is still there, you just need the hardware and protocol support.
> I don't know how it works... it's trivial
Come on now...
And it is easy to monitor, it is just an application concern not a L3-L4 one.
Re: It's always TCP_NODELAY
#233I was curious whether I had to change anything in my applications after reading that so did a bit of research. Both Node.js and Curl use TCP_NODELAY by default from a long time.
Nodejs enabled TCP_NODELAY by default in 2022 v.18. PR: https://github.com/nodejs/node/pull/42163 Changelog entry: https://github.com/nodejs/node/blob/main/doc/changelogs/CHAN...
That's the HTTP module, but it was already moved to NODELAY for the 'net' module in 2015.
Re: It's always TCP_NODELAY
#234Earlier quoted context omitted.
If a socket is closed properly there'll be a FIN and the other side can learn about it by polling the socket. If the network connection is lost due to external circumstances (say your modem crashes) then how would that information propagate from the point of failure to the remote end on an idle connection ? Either you actively probe (keepalives) and risk false positives or you wait until you hear again from the other…
> If the network connection is lost due to external circumstances (say your modem crashes) then how would that information propagate from the point of failure to the remote end on an idle connection ? Observe the line voltage? If it gets cut then you have a problem... > Either you actively probe (keepalives) and risk false positives What false positives? Are you thinking there's an adversary on the other side?
For a given connection, (most) packages might go through (e.g.) 10 links. If one link goes down (or is saturated and dropping packets) the connection is supposed to route around it.
So, except for the links on either of end going down (one end really, if the other is on a “data center” the TCP connection is likely terminated in a “server” with redundant networking) you wouldn't want to have a connection terminated just because a link died.
That's explicitly against the goal of a packed switched network.
Re: It's always TCP_NODELAY
#235Earlier quoted context omitted.
There's a crucial difference in fact, which is that the peer you're defining connectedness to is a single well-defined peer that is directly connected to you, which "The Network" is not. As for the analogy, uh, this ain't water. Monitor the line voltage or the fiber brightness or something, it'll tell you very quickly if the other endpoint is disconnected. It's up to the physical layer to provide a mechanism to detec…
Well, isn't that already how it works? If I physically unplug my ethernet cable, won't TCP-related syscalls start failing immediately?
I ran into this with websockets. At least under certain browser/os pairs you won't ever receive a close event if you disconnect from wifi. I guess you need to manually monitor ping/pong messages and close it yourself after a timeout?
Re: It's always TCP_NODELAY
#236Earlier quoted context omitted.
I think it's a distinction without a difference in this case. You can't know if the reason your water stopped is because the water is shut off, the pipe broke, or it's just slow. When all you have to go on is "I stopped getting packets" the best you can do is give up after a bit. TCP keepsalives do kinda suck and are full of interesting choices that don't seem to have passed the test of time. But they are there and i…
There's a crucial difference in fact, which is that the peer you're defining connectedness to is a single well-defined peer that is directly connected to you, which "The Network" is not. As for the analogy, uh, this ain't water. Monitor the line voltage or the fiber brightness or something, it'll tell you very quickly if the other endpoint is disconnected. It's up to the physical layer to provide a mechanism to detec…
In a packet-switched network there isn't one connection between you and your peer. Even if you had line monitoring that wouldn't be enough on its own to tell you that your packet can't get there -- "routing around the problem" isn't just a turn of phrase. On the opposite end networks are best-effort so even if the line is up you might get stuck in congestion which to you might as well be dropped.
You can get the guarantees you want with a circuit switched network but there's a lot of trade-offs namely bandwidth and self-healing.
Re: It's always TCP_NODELAY
#237Earlier quoted context omitted.
> Fix the apps. Nobody expect magical perf if you do that when writing to files, We write to files line-by-line or even character-by-character and expect the library or OS to "magically" buffer it into fast file writes. Same with memory. We expect multiple small mallocs to be smartly coalesced by the platform.
If you expect a POSIX-y OS to buffer write(2) calls, you're sadly misguided. Whether or not that happens depends on nature of the device file you're writing to. OTOH, if you're using fwrite(3), as you likely should be actual file I/O, then your expectation is entirely reasonable. Similarly with memory. If you expect brk(2) to handle multiple small allocations "sensibly" you're going to be disappointed. If you use mal…
Whether buffering is part of POSIX or not is beside the point. Any modern OS you'll find will buffer write calls in one way or the other. Similarly with memory. Linux waits until accesses page faults before reserving any memory pages for you. My point is that various forms of buffering is everywhere and in practice we do rely on it a whole lot.
Re: It's always TCP_NODELAY
#238Earlier quoted context omitted.
If you expect a POSIX-y OS to buffer write(2) calls, you're sadly misguided. Whether or not that happens depends on nature of the device file you're writing to. OTOH, if you're using fwrite(3), as you likely should be actual file I/O, then your expectation is entirely reasonable. Similarly with memory. If you expect brk(2) to handle multiple small allocations "sensibly" you're going to be disappointed. If you use mal…
Whether buffering is part of POSIX or not is beside the point. Any modern OS you'll find will buffer write calls in one way or the other. Similarly with memory. Linux waits until accesses page faults before reserving any memory pages for you. My point is that various forms of buffering is everywhere and in practice we do rely on it a whole lot.
> Any modern OS you'll find will buffer write calls in one way or the other.
This is simply not true as a general rule. It depends on the nature of the file descriptor. Yes, if the file descriptor refers to the file system, it will in all likelihood be buffered by the OS (not with O_DIRECT, however). But on "any modern OS", file descriptors can refer to things that are not files, and the buffering situation there will vary from case to case.
Re: It's always TCP_NODELAY
#239I've fixed multiple latency issues due to nagle's multiple times in my career. It's the first thing I jump to. I feel like the logic behind it is sound, but it just doesn't work for some workloads. It should be something that an engineer needs to be forced to set while creating a socket, instead of letting the OS choose a default. I think that's the main issue. Not that it's a good / bad option but that there is a se…
Not when creating a socket - when sending data. When sending data, you should indicate whether this data block prefers high throughput or low latency.
Re: It's always TCP_NODELAY
#240Earlier quoted context omitted.
I think that's more two independent byte streams. You want low latency but what is transfered doesnt really impact the other side, you just constantly want to push the next frame
Thanks, that makes sense! It's interesting that it's very much an interactive experience for the end-user. But for the logic of the computer, it's not interactive at all. You can make the contrast even stronger: if both video streams are transmitted over UDP, you don't even need to sent ACKs etc. To be truly one-directional from a technical point of view. Then compare that to transferring a file via TCP. For the user…
Video calls indeed almost always use UDP. TCP retransmission isn't really useful since by the time a retransmitted packet arrives it's too old to display. Worse, a single lost packet will block a TCP stream. Sometimes TCP is the only way to get through a firewall, but the experience is bad if there's any packet loss at all.
VC systems do constantly send back packet loss statistics and adjust the video quality to avoid saturating a link. Any buffering in routers along the way will add delay, so you want to keep the bitrate low enough to keep buffers empty.