io_uring doesn't support getdents though. so the primary benefit is bulk statting (ls -l).
It'd be nice if we could have a getdents in flight while processing the results of the previous one.
lsr: ls with io_uring
41–50 of 177 posts
Re: lsr: ls with io_uring
#42Earlier quoted context omitted.
> Why does this require inventing lsr as an alternative to ls instead of making ls use io_uring? Good luck getting that upstreamed and accepted. The more foundational the tools (and GNU coreutils definitely is foundational), the more difficult that process will be. Releasing a standalone utility makes iteration much faster, partially because one is not bound to the release cycles of distributions.
> Releasing a standalone utility makes iteration much faster, partially because one is not bound to the release cycles of distributions. which certainly is a valid way or prioritizing. similarly, distros/users may prioritize stability, which means the theoretical improvement would now be stuck in not-used-land. the value of software appears when it's run, not when it's written
> the value of software appears when it's run, not when it's written
Have you ever tried to contribute to open source projects?
The question was why wouldn't someone writing software not take the route likely to end in rejection/failure. I don't know about you, but if I write software, I am not going to write it for a project whose managers will make it difficult for my PR to be accepted, and that 99% likely it never will be.
I will always contribute to the project likely to appreciate my work and incorporate it.
I'll share an anecdote: I got involved with a project, filed a couple PRs that were accepted (slowly), and then I talked about refactoring something so it could be tested better and wasn't so fragile and tightly coupled to IO. "Sounds great" was the response.
So I did the refactor. Filed a PR and asked for code review. The response was (after a long time waiting) "thanks but no, we don't want this." PR closed. No feedback, nothing.
I don't even use the software anymore. I certainly haven't tried to fix any bugs. I don't like being jerked around by management, especially when I'm doing it for free.
(For the record, I privately forked the code and run my own version that is better because by refactoring and then writing tests, I discovered a number of bugs I couldn't be arsed to file with the original project.)
Re: lsr: ls with io_uring
#43I wonder how it performs against an NFS server with lots of files, especially one over a kinda-crappy connection. Putting an unreliable network service behind blocking POSIX syscalls is one of the main reasons NFS is a terrible design choice (as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder), but I wonder if io_uring mitigates the bad parts somewhat.
> as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder
Theoretically "intr" mounts allowed signals to interrupt operations waiting on a hung remote server, but Linux removed the option long ago[1] (FreeBSD still supports it)[2]. "soft" might be the only workaround on Linux.
[1]: https://man7.org/linux/man-pages/man5/nfs.5.html
[2]: https://man.freebsd.org/cgi/man.cgi?query=mount_nfs&sektion=...
Re: lsr: ls with io_uring
#44Love it. I'm trying to understand why all command line tools don't use io_uring. As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak. If I use tools with aio or io_uring I get 1005MB/s. I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.
io_uring is a security nightmare.
Re: lsr: ls with io_uring
#45Re: lsr: ls with io_uring
#46Love it. I'm trying to understand why all command line tools don't use io_uring. As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak. If I use tools with aio or io_uring I get 1005MB/s. I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.
io_uring is a security nightmare.
How so?
Re: lsr: ls with io_uring
#47Really interesting, the difference is real though I would just hope that some better coloring support could be added because I have "eza --icons=always -1" command set as my ls and it looks really good, whereas when I use lsr -1, yes the fundamental thing is same, the difference is in the coloring. Yes lsr also colors the output but it doesn't know as many things as eza does For example .opus will show up as a music…
[flagged]
Re: lsr: ls with io_uring
#48Earlier quoted context omitted.
In addition, the author might not want to sign away their rights to the FSF.
What on earth are you talking about? Why would this be the case?
Depending on the implementation (and I don't know which `ls` is being referred to), modifying `ls` might mean modifying an FSF project which require copyright assignment as a condition of patch submissions.
Re: lsr: ls with io_uring
#49Earlier quoted context omitted.
io_uring is a security nightmare.
How so?
This is a good read on the topic: https://chomp.ie/Blog+Posts/Put+an+io_uring+on+it+-+Exploiti...
Re: lsr: ls with io_uring
#50The times seem sublinear, 10k files is less than 10x 1k files.
I remember getting in to a situation during the ext2 and spinning rust days where production directories had 500k files. ls processes were slow enough to overload everything. ls -F saved me there.
And filesystems got a lot better at lots of files. What filesystem was used here?
It's interesting how well busybox fares, it's written for size not speed iirc?