I couldn't find the bit where the original performance claim was refuted (or not). Was it one of the listed options?
How efficient can cat(1) be?
11–20 of 51 posts
Re: How efficient can cat(1) be?
#12Slightly out of topic: > There have been a few initiatives in recent years to implement new a new userspace base system for Linux distributions as an alternative to the GNU coreutils and BusyBox. Have there been? And why?
Why indeed! Recent versions of GNU cat (and other coreutils) include those optimizations.
https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...
Re: How efficient can cat(1) be?
#13A bit of a tangent, there are few instances of "do-while" that I've ever ever written and not removed soon after. In practice, I've found that the looping situations that don't easily match the "for (int i = 0; i do { splice(from stdin...); if (A) handle_a(); goto out; splice(to stdout...); if (B) handle_b(); goto out; } while (nwritten > 0); // do we need some kind of handle_c()?? out: ... Why make a special case fo…
First of all: Your code is missing braces around the `if` blocks - the `goto out` would be run unconditionally.
But anyway, the case for `goto` here is that it jumps immediately to the cleanup that needs to happen always.
If you put something between the loop and that, `break` would jump before that and also execute that.
Yes, this is a workaround for C's lack of automatic cleanup (RAII, garbage collection, python's `with` or whatever).
Re: How efficient can cat(1) be?
#14While it's certainly just example code, the initial sendfile version is badly buggy.
/* Fall back to traditional copy if the spliced version fails. */
if (!spliced_copy(srcfd))
copy(srcfd);
Which means sendfile could output half the data, fail for some reason, and depending on why it failed, the fallback copy read/write will do bad things. for example, output the same data again, or more likely, skip data. Since they are just reading from the fd as it now exists after sendfile failing, it is most likely to skip data but pretend it completed successfully.
Normally, cat would just fail in that situation, as this should - it should not retry the copy when sendfile returns EINVAL or ENOSYS
This is what you get for transforming error codes into booleans :)
(I guess errno will still be set, and they could still check it here, but ugh)
[1] This is why the man page says: Applications may wish to fall back to read(2)/write(2) in the case where sendfile() fails with EINVAL or ENOSYS.
Re: How efficient can cat(1) be?
#15While it's certainly just example code, the initial sendfile version is badly buggy. /* Fall back to traditional copy if the spliced version fails. */ if (!spliced_copy(srcfd)) copy(srcfd); The thing that they are trying to avoid is sendfile failing due to inability to mmap the the fd. But they don't check for that (it would return EINVAL), and in fact, by converting the error code to boolean, destroy the ability to…
Should sendfile(2) not report error if it managed to send a non-zero amount of data? IIRC write(2) behaves like that.
Re: How efficient can cat(1) be?
#16how about ptracing into your target process and dup2()'ing your FD across?
infinitely fast cat
Re: How efficient can cat(1) be?
#17I couldn't find the bit where the original performance claim was refuted (or not). Was it one of the listed options?
The original performance claim was about https://vimuser.org/cat.c.txt.
Which just does read/write - so it's the same as the "cat-simple" example, which is the slowest listed.
GNU cat [0] does copy_file_range if it can and falls back to a read/write loop otherwise, so it's unlikely to be much slower (possibly some overhead from argument parsing, but that's just a constant).
So the performance claims are wrong.
[0]: https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...
Re: How efficient can cat(1) be?
#18While it's certainly just example code, the initial sendfile version is badly buggy. /* Fall back to traditional copy if the spliced version fails. */ if (!spliced_copy(srcfd)) copy(srcfd); The thing that they are trying to avoid is sendfile failing due to inability to mmap the the fd. But they don't check for that (it would return EINVAL), and in fact, by converting the error code to boolean, destroy the ability to…
Should sendfile(2) not report error if it managed to send a non-zero amount of data? IIRC write(2) behaves like that.
I don't honestly remember all of the write semantics - I thought it will only not report error if it gets interrupted by a signal handler after it wrote something.
In case of true error, i thought it always return an error.
write does have a weird error-checking semantic - you can get it to check for a bunch of errors without writing data by using a count of 0.
At least as documented, sendfile does not have any of these semantics - it only returns number of bytes written if the transfer was successful. Otherwise, you can't tell how far it made it :)
Re: How efficient can cat(1) be?
#19This was a good read! The missing hyperlinks: - https://man7.org/linux/man-pages/man2/sendfile.2.html - https://man7.org/linux/man-pages/man2/splice.2.html (Funny how used I've gotten to "hypertext", I was quite irritated I couldn't click those function names.)
Pinfo makes regular man pages navigable.
And makes info pages pleasant to view.
Re: How efficient can cat(1) be?
#20A bit of a tangent, there are few instances of "do-while" that I've ever ever written and not removed soon after. In practice, I've found that the looping situations that don't easily match the "for (int i = 0; i do { splice(from stdin...); if (A) handle_a(); goto out; splice(to stdout...); if (B) handle_b(); goto out; } while (nwritten > 0); // do we need some kind of handle_c()?? out: ... Why make a special case fo…
First of all: Your code is missing braces around the `if` blocks - the `goto out` would be run unconditionally. But anyway, the case for `goto` here is that it jumps immediately to the cleanup that needs to happen always. If you put something between the loop and that, `break` would jump before that and also execute that. Yes, this is a workaround for C's lack of automatic cleanup (RAII, garbage collection, python's…
> Your code is missing braces around the `if` blocks
Sure. It's obviously a sketch.
> If you put something between the loop and that, `break` would jump before that and also execute that.
Sure. But I rarely can see a need to make it so complicated (not in this case anyway). If you need multiple distinct cleanup sections that can't be inlined, then label them all (or put them in separate procedures) and jump to them explicitly. In my example, there is no need for any labels at all.
> this is a workaround for C's lack of automatic cleanup (RAII, garbage collection, python's `with` or whatever).
No need for workarounds here.