31–40 of 46 posts
Earlier quoted context omitted.
Well POSIX only defines behavior, not performance. Every platform and OS will have its own performance idiosyncracies.
How on earth would POSIX define performance of something like pipes?
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…
When you say the performance was hideous, are you referring to I/O after the pipe is already connected/open, or before? The former would he surprising, but the latter not - opening and closing a ton of pipes is not something you'd expect an OS to be optimized for - and it would be somewhat surprising if your use case requires the latter.
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…
> The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. When you say the performance was hideous, are you referring to I/O after the pipe is already connected/open, or before? The former would he surprising, but the latter not - opening and closing a ton of pipes is not something you'd expect an OS to be optimized for - and it would be somew…
Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.
shared memory provides the lowest latency, but you still need to deal with task wakeup, which is usually done via futexs. Google was working on a FUTEX_SWAP call for linux which would have allowed direct handover from one task to another, not sure what happened to that.
Earlier quoted context omitted.
AFAIK, neither OpenBSD nor NetBSD has sendfile. On FreeBSD, I think you're correct regarding it being file-to-socket only.
Indeed, if I'm not mistaken Netflix at least used to use (and commit to kernel) FreeBSD on content servers because of its superior sendfile performance
Earlier quoted context omitted.
> The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. When you say the performance was hideous, are you referring to I/O after the pipe is already connected/open, or before? The former would he surprising, but the latter not - opening and closing a ton of pipes is not something you'd expect an OS to be optimized for - and it would be somew…
Literally just having spare listening sockets, ready for incoming connections (and obv. not busy-waiting on them). Just reducing to the number actually in-use was the biggest speed-up - it was like Windows was busy-waiting internally for new connections (it wasn't a huge number either, something like 8 or 12).
And yeah, that seems more or less what I expected. The implementation is probably optimized for repeated I/O on established connections, not repeated unestablished ones. Which would be similar to filesystem I/O on Windows in that way - it's optimized for I/O on open files (especially larger ones), not for repeatedly opening and closing files (especially small ones). It makes me wonder what kinds of use cases require repeated connections on named pipes.
If the performance is comparable to Linux's after the connection, then I think that's important to note - since that's what matters to a lot of applications.
Earlier quoted context omitted.
Literally just having spare listening sockets, ready for incoming connections (and obv. not busy-waiting on them). Just reducing to the number actually in-use was the biggest speed-up - it was like Windows was busy-waiting internally for new connections (it wasn't a huge number either, something like 8 or 12).
By "spare listening sockets" do you mean having threads on the server calling ConnectNamedPipe? A bit confused by your terminology since these aren't called listening sockets. (You're not referring to socket() or AF_UNIX, right?) And yeah, that seems more or less what I expected. The implementation is probably optimized for repeated I/O on established connections, not repeated unestablished ones. Which would be simil…
Earlier quoted context omitted.
By "spare listening sockets" do you mean having threads on the server calling ConnectNamedPipe? A bit confused by your terminology since these aren't called listening sockets. (You're not referring to socket() or AF_UNIX, right?) And yeah, that seems more or less what I expected. The implementation is probably optimized for repeated I/O on established connections, not repeated unestablished ones. Which would be simil…
Yes, it was indeed using ConnectNamedPipe - just had a look at the code (which I can't share) to refresh my memory. The main problem was traced to setup delays in WaitForSingleObject()/WaitForMultipleObjects(); we fixed it as above (once all sessions were connected there were no spares left, so no problems), actual throughput was noted as quite inferior to linux but more than enough for our application so we left it…
I would intuitively expect throughout (once all clients have connected) to be similar to on Linux, unless the Linux side uses syscalls like vmsplice() - but not sure, I've never tried benchmarking.
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…