Earlier quoted context omitted.
A directory is a file like anything else that contains a map of names to inodes. If you're trying to add or remove mappings (create or delete files), then clearly some synchronization must occur or the contents of the file will contain garbage. In theory you could get away with a very small critical section that says "lock bytes N through M" of the file, but then how do you deal with disk block alignment (i.e. two pa…
> A directory is a file like anything else that contains a map of names to inodes. If you're trying to add or remove mappings (create or delete files), then clearly some synchronization must occur or the contents of the file will contain garbage. this synchronization is handled for you by the fs, specifically the fs cache inode alignment and errors are managed by this intermediating layer your benchmarks are not demo…
The fastest rm command and one of the fastest cp commands
81–83 of 83 posts
Re: The fastest rm command and one of the fastest cp commands
#82Earlier quoted context omitted.
> A directory is a file like anything else that contains a map of names to inodes. If you're trying to add or remove mappings (create or delete files), then clearly some synchronization must occur or the contents of the file will contain garbage. this synchronization is handled for you by the fs, specifically the fs cache inode alignment and errors are managed by this intermediating layer your benchmarks are not demo…
Except they are and your claims are trivial to disprove: simply run the benchmarks under perf. You'll find that most of the time is spent on the rwsem which is described here onwards: https://www.kernel.org/doc/html/latest/filesystems/path-look...
the fs cache does most/all of the optimizations you're doing manually
bypassing the fs cache is highly atypical for user-space code
Re: The fastest rm command and one of the fastest cp commands
#83> The key insight is that file operations in separate directories don’t (for the most part) interfere with each other, enabling parallel execution. i'm clearly missing something here parallel execution helps when operations are cpu bound file operations are (almost always) io bound and totally unclear how directories represent an "interference" boundary bizarre
What I would suppose is that it reduces the amount of redundant IO where a directory is edited, flushed to disk, and then later updated again. If all these updates happen in a batch there will be less IO overall.