Kind of fascinating that slashing syscalls by ~35x (versus the `ls -la` benchmark) is "only" worth a 2x speedup
Can someone try this in a virtual machine? I’d imagine syscalls there to be more expensive.
lsr: ls with io_uring
161–170 of 177 posts
Re: lsr: ls with io_uring
#162A relevant task in uutils https://github.com/uutils/coreutils/issues/4362 describes how io_uring universally helps in almost any disk operation util.
Re: lsr: ls with io_uring
#163Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring
Nice writeup. I suspect you're measuring the cost of abstraction. Specifically, routines that can handle lots of things (like locale based strings and utf8 character) have more things to do before they can produce results. This was something I ran into head on at Sun when we did the I18N[1] project. In my experience there was a direct correlation between the number of different environments where a program would "jus…
> the 12 bits of meta data that God intended
Naw, that's Dennis Ritchie. You're thinking of the other white-bearded guy that hangs out in heaven.
Re: lsr: ls with io_uring
#164Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring
Nice writeup. I suspect you're measuring the cost of abstraction. Specifically, routines that can handle lots of things (like locale based strings and utf8 character) have more things to do before they can produce results. This was something I ran into head on at Sun when we did the I18N[1] project. In my experience there was a direct correlation between the number of different environments where a program would "jus…
Other people attribute i18n to DEC circa 1985.
Re: lsr: ls with io_uring
#165Earlier quoted context omitted.
> Have you ever tried to contribute to open source projects? yes, and it was often painful enough to make me consider very well wether I want to bother contributing. I can only imagine how terrible the experience must be at a core utility such as ls. > The question was why wouldn't someone writing software not take the route likely to end in rejection/failure Obviously they wouldn't - in my comment I assumed that the…
> The older OSS projects become, the more they fossilize too - and that makes it more annoying to contribute. Newer ones can be just as braindead, if they came out of some commercial entity. CLAs and such.
That CLA is a curious take, as the old, avowedly non-commercial, GNU foundational tools such as the ones we're talking about like "ls" in coreutils, have always required a kind of strong CLA to be signed from the very beginning, even when they wee new, nimble and fun to contribute to.
Their kind of CLA was designed to uphold community and openness values more strongly than GPL alone, by helping GNU to pursue GPL violators through the law, to fource vendors to release source code when GPL code was shipped in products..
So I've never understood the blanket "don't like regardless of what it says" attitude to CLAs and such.
Surely it should depend on what the CLA says?
Some people object to CLAs that grant upstream less rights than BSD/MIT/Apache licenses grant upstream by defaut. ("No way, the CLA lets them make. a private, commercial fork of my code!"). Yet the same people contribute enthusiastically to BSD/MIT/Apache projects with exactly the same upstream property ("I don't mind that the license let's them make a private, commercial fork of my code!").
Re: lsr: ls with io_uring
#166Earlier quoted context omitted.
The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff. It didn't require every existing tool, such as ls, to deal with things like the server rebooting while listing a directory. (The original NFS protocol is stateless, so clients can survive server reboots.) What does vi do when the server hosting the file you…
> The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff I don't agree that it was a reasonable tradeoff. Making an unreliable system emulate a reliable one is the very thing I find to be a bad idea. I don't think this is unique to NFS, it applies to any network filesystem you try to present as if it's a local o…
Sure, at some point you have to let the abstraction leak. At the time Sun designed NFS, you could basically count on the fact that the server was some Solaris machine capable of multiple years of uptime on your LAN. Filesystems never made the transition to running over the internet because that was too unreliable and POSIX didn't really provide the right interfaces to expose that.
We're coming to the end of the road with that generation of OS design - MacOS is still Unix and thinks that it's running on a VAX. There's a reason why Macbooks don't come with a 5G modem: because programs would have to be aware of the underlying network. That's why it's inevitable that we'll move to something like IOS (or Android), because every program that uses the network has to handle not only failures but situations like being in flight mode or running on low-bandwidth mobile networks.
Re: lsr: ls with io_uring
#167Re: lsr: ls with io_uring
#168This is awesome for learning and a funzies app. Would be nice to see it moved into some mainstream distro, to be honest.
That said, in over 35 years of using `ls`, never once have I bemoaned the speed of it.
Re: lsr: ls with io_uring
#169This is awesome for learning and a funzies app. Would be nice to see it moved into some mainstream distro, to be honest. That said, in over 35 years of using `ls`, never once have I bemoaned the speed of it.
Using ls in a dir with millions of files may stall a good while.
Re: lsr: ls with io_uring
#170Earlier quoted context omitted.
Nice writeup. I suspect you're measuring the cost of abstraction. Specifically, routines that can handle lots of things (like locale based strings and utf8 character) have more things to do before they can produce results. This was something I ran into head on at Sun when we did the I18N[1] project. In my experience there was a direct correlation between the number of different environments where a program would "jus…
Other people attribute i18n to DEC circa 1985. http://www.i18nguy.com/origini18n.html
I did not mean to imply I invented it, that was just the first time I had seen that form of shorthand. Presumably whomever at Sun who called it that was no doubt aware of other efforts, I was but a lowly entry level programmer in the systems group. That was long before I had acquired the power to name things at Sun :-)