Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux Pipes Are Slow

qsantos.fr

Post reply on HN
Print view View original

81–90 of 171 posts

Re: Linux Pipes Are Slow

#81

Post by sebstefan »

samastur wrote: post #76
Someone wrote: post #40

Earlier quoted context omitted.

This isn’t code in some project that will run only a few billion times in its lifetime; it is used frequently on millions, if not billions, of computers. Because of that, it is economical to spend lots of time optimizing it, even if it only makes the code marginally more efficient.

That's not how economics works. If 100 million people each save 1 cent because of your work, you saved 1 million in total, but in practice nobody is observably better off.

So? It doesn't need to be visible to be worth optimizing?

Re: Linux Pipes Are Slow

#83

Post by mort96 »

0xbadcafebee wrote: post #13

Calling Linux pipes "slow" is like calling a Toyota Corolla "slow". It's fast enough for all but the most extreme use cases. Are you racing cars? In a sport where speed is more important than technique? Then get a faster car. Otherwise stick to the Corolla.

I mean why waste CPU time moving data between buffers when you could get the same semantics and programming model without wasting that CPU time?

Re: Linux Pipes Are Slow

#85

Post by rwmj »

Be interesting to see a version using io_uring, which I think would let you pre-share buffers with the kernel avoiding some copies, and avoid syscall overhead (though the latter seems negligible here).

Re: Linux Pipes Are Slow

#86

Post by qsantos »

rwmj wrote: post #85

Be interesting to see a version using io_uring, which I think would let you pre-share buffers with the kernel avoiding some copies, and avoid syscall overhead (though the latter seems negligible here).

That sounds like a good idea!

Re: Linux Pipes Are Slow

#87

Post by h0p3 »

samastur wrote: post #76
Someone wrote: post #40

Earlier quoted context omitted.

This isn’t code in some project that will run only a few billion times in its lifetime; it is used frequently on millions, if not billions, of computers. Because of that, it is economical to spend lots of time optimizing it, even if it only makes the code marginally more efficient.

That's not how economics works. If 100 million people each save 1 cent because of your work, you saved 1 million in total, but in practice nobody is observably better off.

There are people whose lives are improved by having an extra cent to spend. Seriously. It is measurable, observable, and real. It might not have a serious impact on the vast majority of people, but there are people who have very, very little money or have found themselves on a tipping point that small; pinching pennies alters their utility outcomes.

Re: Linux Pipes Are Slow

#88

Post by qsantos »

fatcunt wrote: post #73

> I do not know why the JMP is not just a RET, however. This is caused by the CONFIG_RETHUNK option. In the disassembly from objdump you are seeing the result of RET being replaced with JMP __x86_return_thunk. https://github.com/torvalds/linux/blob/v6.1/arch/x86/include... https://github.com/torvalds/linux/blob/v6.1/arch/x86/lib/ret... > The NOP instructions at the beginning and at the end of the function allow ftrac…

Thanks a lot for the information! I was not quite sure what to look for in this case. I have added in note in the article.

Re: Linux Pipes Are Slow

#89

Post by qsantos »

ismaildonmez wrote: post #70
qsantos wrote: post #42

Earlier quoted context omitted.

I'll write an article on the flamegraphs specifically, but to get the data, just follow Julia's article! https://jvns.ca/blog/2017/03/19/getting-started-with-ftrace/

Could you clarify how are you testing the speed of the first example where you are not writing anything to stdout? Thanks.

For the first Rust program, where I just write to memory, I just use the time utility when running the program from zsh. Then, I divide the number of bytes written by the number of seconds elapsed. That's why it's not an infinite loop ;)

Re: Linux Pipes Are Slow

#90

Post by qsantos »

0xbadcafebee wrote: post #13

Calling Linux pipes "slow" is like calling a Toyota Corolla "slow". It's fast enough for all but the most extreme use cases. Are you racing cars? In a sport where speed is more important than technique? Then get a faster car. Otherwise stick to the Corolla.

To be frank, this is more of a pretext to understand what pipes and vmsplice do exactly.
Post reply on HN