Live data from Hacker News

How fast are Linux pipes anyway? (2022)

mazzo.li

21–30 of 46 posts

Re: How fast are Linux pipes anyway? (2022)

#21
post #16

Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.

Would be helpful to know what your problem is with AF_UNIX at the moment. Is it lacking in features you want? Is it higher latency than you'd want? Is the server/client socket API style not appropriate for your use-case?

Re: How fast are Linux pipes anyway? (2022)

#22
post #21
post #16

Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.

Would be helpful to know what your problem is with AF_UNIX at the moment. Is it lacking in features you want? Is it higher latency than you'd want? Is the server/client socket API style not appropriate for your use-case?

Well, it’s probably fine but, it’s an audio application where metering (not audio) is delivered from a control plane process to a UI process. Lower latency is better. But haven’t measured it.

Re: How fast are Linux pipes anyway? (2022)

#23
post #7

Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…

Some years back Windows added AF_UNIX sockets, I wonder how those would perform relative to Win32 pipes. My guess is better.

Seems to reportedly be slightly faster in a few cases, but nothing particularly dramatic https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tc...

Re: How fast are Linux pipes anyway? (2022)

#24
post #16

Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.

What are Doors, it's too common a word to Google.

Re: How fast are Linux pipes anyway? (2022)

#25

FWIW there is readv() / writev(), splice(), sendfile(), funopen(), and io_buffer() as well. splice() is great when transferring data between pipes and UNIX sockets with zero-copy, but it is Linux-only. splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra bu…

> splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal.

Proper use of io_uring should finally have it beat or at least matched.

Re: How fast are Linux pipes anyway? (2022)

#26
post #16

Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.

What are Doors, it's too common a word to Google.

Lightweight IPC invented by Sun.

https://en.m.wikipedia.org/wiki/Doors_(computing)

Look for Doors solaris and there are quite a few articles.

Re: How fast are Linux pipes anyway? (2022)

#29
post #7

Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. Some years later this got revisited due to needing to use the same thing under C# on Win10 and while…

Did you find that you needed interprocess communication to replace the gap?

pipes are a form of interprocess communication :) I guess you meant shared memory?

Re: How fast are Linux pipes anyway? (2022)

#30

Earlier quoted context omitted.

Did you find that you needed interprocess communication to replace the gap?

pipes are a form of interprocess communication :) I guess you meant shared memory?

Yes. Yeah, you're right. Sockets could also be used, but I guess when I think of IPC, I generally think of shared memory.
Post reply on HN