Live data from Hacker News

How efficient can cat(1) be?

ariadne.space

21–30 of 51 posts

Re: How efficient can cat(1) be?

#21
post #2

This was a good read! The missing hyperlinks: - https://man7.org/linux/man-pages/man2/sendfile.2.html - https://man7.org/linux/man-pages/man2/splice.2.html (Funny how used I've gotten to "hypertext", I was quite irritated I couldn't click those function names.)

Pinfo makes regular man pages navigable.

I'm having a great Linux day! TIL about 'splice' and 'pinfo'. Thanks!

Re: How efficient can cat(1) be?

#22

A bit of a tangent, there are few instances of "do-while" that I've ever ever written and not removed soon after. In practice, I've found that the looping situations that don't easily match the "for (int i = 0; i do { splice(from stdin...); if (A) handle_a(); goto out; splice(to stdout...); if (B) handle_b(); goto out; } while (nwritten > 0); // do we need some kind of handle_c()?? out: ... Why make a special case fo…

"do ... while" maps neatly into Assembly for me. I think it is meant as a bridge between low level thinking and structured programming.

Re: How efficient can cat(1) be?

#23
post #12
post #10

Slightly out of topic: > There have been a few initiatives in recent years to implement new a new userspace base system for Linux distributions as an alternative to the GNU coreutils and BusyBox. Have there been? And why?

Why indeed! Recent versions of GNU cat (and other coreutils) include those optimizations. https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...

To be honest, GNU coreutils is not a good alternative to GNU coreutils. :-P

Re: How efficient can cat(1) be?

#24
post #10

Slightly out of topic: > There have been a few initiatives in recent years to implement new a new userspace base system for Linux distributions as an alternative to the GNU coreutils and BusyBox. Have there been? And why?

I heard only about this one:

https://github.com/uutils/coreutils ("Cross-platform Rust rewrite of the GNU coreutils")

Re: How efficient can cat(1) be?

#26
post #25

It would be nice if user programs didn't have to jump through loops like this. It would be ideal if the kernel made the naive implementation work efficiently.

How would you suggest the kernel would accomplish that feat? If the user calls read() on a file descriptor, what can the kernel do except... you know... actually read from it and copy the data to user space?

Re: How efficient can cat(1) be?

#27
post #26
post #25

It would be nice if user programs didn't have to jump through loops like this. It would be ideal if the kernel made the naive implementation work efficiently.

How would you suggest the kernel would accomplish that feat? If the user calls read() on a file descriptor, what can the kernel do except... you know... actually read from it and copy the data to user space?

I know it wouldn't be easy. But the goal could be to let users write simple programs that say what they mean. Since many programs do: while(!eof) { read(); write(); } that could be optimized similar to the way a compiler optimizes known idioms. Perhaps it could notice a well know sequence of calls and avoid userspace/kernels space copies. But, yeah, very non-trivial.

Re: How efficient can cat(1) be?

#28
From my experience, it will never be so efficient that someone smarter than me doesn't publicly shame me for winning the "useless use of cat" award on a forum where I ask for help.

28 years later and I'm still sore I asked for help as a 15 year old that one time. Very effective way to teach a new user.

Re: How efficient can cat(1) be?

#29
post #26
post #25

It would be nice if user programs didn't have to jump through loops like this. It would be ideal if the kernel made the naive implementation work efficiently.

How would you suggest the kernel would accomplish that feat? If the user calls read() on a file descriptor, what can the kernel do except... you know... actually read from it and copy the data to user space?

It _could_ look at the instructions following the read call and do a sort of high-level software macro-op fusion (https://en.wikichip.org/wiki/macro-operation_fusion)

That already would be going beyond the duty for a jitting virtual machine, so I don’t think you can expect that from a CPU. I also likely would make lots of programmers uneasy if their OS does that sort of thing.

For a ‘real’ CPU, I also fear handling all the edge cases would be horrific (the program may do a read-write pair of calls, but how do you ascertain it doesn’t read that data later? What if the read call tries to read into unwritable memory? What if the program is being traced, and read calls are being logged? Etc)

Re: How efficient can cat(1) be?

#30

From my experience, it will never be so efficient that someone smarter than me doesn't publicly shame me for winning the "useless use of cat" award on a forum where I ask for help. 28 years later and I'm still sore I asked for help as a 15 year old that one time. Very effective way to teach a new user.

[deleted]
Post reply on HN