Git ls-files is Faster Than Fd and Find
1–10 of 80 posts
Re: Git ls-files is Faster Than Fd and Find
#2> source code management system (SCM), it’s main business3 is not to help you list your files!
This seems utterly bizarre statement. What does author imagine SCM dies? Stuff, but listing the files that are the source it is managing is up there.
Like saying FPS is a game its not pushing triangles through GPU.
The larger purpose is facilitated by and requires the lessor purpose.
Re: Git ls-files is Faster Than Fd and Find
#3Small typo occurring two times: git ls-file instead of git ls-files.
Thanks for bringing it up, I didn't know about the command. I also might have a use case for it.
Re: Git ls-files is Faster Than Fd and Find
#4Well, first doing `find > .my-index` and then measuring `cat .my-index` would give you even better results... I don't find it noteworthy that reading from an index is faster than actually recursively walking the filesystem.
Re: Git ls-files is Faster Than Fd and Find
#5One thing I’ve never understood about Linux filesystems: given how small and bounded the sets of directories and directory entries are, why is filesystem traversal not instantaneous?
Re: Git ls-files is Faster Than Fd and Find
#6Well, first doing `find > .my-index` and then measuring `cat .my-index` would give you even better results... I don't find it noteworthy that reading from an index is faster than actually recursively walking the filesystem.
Yes. I once wrote a tool which spends a lot of time traversing the filesystem and to my surprise in one scenario most of its time is spent on-CPU (!) in kernel-mode in the readdir_r(2) syscall implementation. I still haven't dived into what it's doing on the CPU, but it sure is interesting.
Re: Git ls-files is Faster Than Fd and Find
#7One thing I’ve never understood about Linux filesystems: given how small and bounded the sets of directories and directory entries are, why is filesystem traversal not instantaneous?
Lack of caches I'd assume. Not because it's not easy, but because no one has taken the time to implement it.
Re: Git ls-files is Faster Than Fd and Find
#8One thing I’ve never understood about Linux filesystems: given how small and bounded the sets of directories and directory entries are, why is filesystem traversal not instantaneous?
Also, why can tab-completion in Bash hang my shell?
Re: Git ls-files is Faster Than Fd and Find
#9One thing I’ve never understood about Linux filesystems: given how small and bounded the sets of directories and directory entries are, why is filesystem traversal not instantaneous?
Lack of caches I'd assume. Not because it's not easy, but because no one has taken the time to implement it.
They say there are two hard problems in computer science: cache invalidation, naming, and off-by-one errors.
Re: Git ls-files is Faster Than Fd and Find
#10One thing I’ve never understood about Linux filesystems: given how small and bounded the sets of directories and directory entries are, why is filesystem traversal not instantaneous?
What you describe is an over-specialized optimization that very few users would benefit from, but would still introduce significant complexity.
Linux already transparently caches filesystem metadata. You already get a good speedup if you attempt the same directory walk twice, and not much have changed in the filesystem.