>
I should probably have phrased this differently, "kernel file system layer integration" maybe. The relevant (and presently, I believe, lacking part) in linux would be VFS. It also relies on applications making consistent use of xattrs for some functionality, something that does also not happen on linux.But that's not how any other those services work - including Spotlight.
> I'm not a file system expert, but I'm increasingly wondering if your confident pronouncements are backed up by sufficient knowledge what you're talking about.
I appreciate your frustration but the problem here is that you keep conflating multiple different technologies and not understanding the distinction I'm trying to make between each of them. I admit I'm not the best at explaining complex technologies (though I wouldn't say the stuff we're talking about is particularly complex) so maybe this conversation is better left to yourself to do some independent research because there is clearly a language gap between what I'm trying to describe and what you're apparently reading.
But the crux of it is you seem to think Spotlight stores all of it's data in the file system itself and is unique in that regard. That isn't true on both counts:
1. Spotlight will use a separate database - not xattr - to store it's indexes.
2. Every tool akin to Spotlight (including Krunner) does the same
There is the caveat that some of the searchable parameters in Spotlight obviously would be in the file system as well as Spotlights database - which might be where you're getting confused? But not everything you described would by xattr and Spotlight itself wouldn't be running slow file system scans to return it's results when it could instead use a local cached database (as I described above) with indexed fields against several parameters rather than just the inode number (which I'll get into later).
You also seem to think that inotify and/or fevents count as "file system integration". It does not. They are completely separate APIs. Whether they're backed by a kernel syscall is completely besides the point because they're not part of the file system ABI. Thus they're not actually tied to the file system itself (ie Spotlight can then work against any file system rather than just apfs).
> Let me try again: with default settings my high spec linux box ground to unusable state (and no I'm not making it up) frequently enough that I got sick off it. So unlike you (somewhat rudely) continue to imply, I'm not some bozo who randomly screwed around with system settings he didn't grok on a whim and then started whining after everything broke.
But you are over commiting resources to virtual machines then moaning when it grinds to a halt. Which isn't any better than tinkering with kernel parameters and making the same complaints.
> Yeah, you do because that way it stays around when you copy, move or archive the file.
That's what fevents is for ;)
By the way, even the file system doesn't index files by file name nor path. Every file system object (files, directories, TTYs, etc) on UNIX and Linux is just an inode. So even the file name and path itself is just metadata stored against the inode. The kernel itself doesn't understand file names, it just passes node indexes around and your file system driver will return metadata such as file name - if requested - by the calling userspace tool. That's how it works at a low level - even though file names and paths feel like a first class parameter in the userspace tools we use.
The reason you don't want too much metadata in the file system itself (eg xattr) is because it slows down file system operations. In fact many GUI platforms intentionally store extended attributes in hidden (technically just dot-prefixed because there isn't actually a "hidden" attribute on UNIX) for that reason. Partly that reason anyway - the other part is because not all filesystems support xattr. Which is actually another reason Spotlight wouldn't want to use xattr.
> Also, whilst I agree that storing search indexes and everything directly in the file system is probably not ideal, there is historical precedent of a systems that did exactly that, fairly successfully from what i hear (BeOS/BFS).
I did run BeOS but I can't remember much about BFS so I'm not going to comment on that specifically, however the other systems were split between two camps:
1. They either stored extended attributes in hidden files or directories - such as .Directory (KDE), .DS_Store (OSX), desktop.ini (Windows) - or
2. instead of a traditional file system layout they will have what is ostensibly be a fully fledged RDBMS. Those tended to be exclusive to mainframes but Microsoft was experimenting with a similar approach with WinFS in Longhorn (I think it was called?). However it was eventually canned due to it's shitty performance.
That's at least the hysterical precedence of storing super detailed meta-information. Historically the stuff that appeared to be stored as xattr were often just read from the file data itself (eg image sizes might be read from the JPEG headers). In fact in the 90s it was common for some platforms to identify what the type of file by literally reading the first few bytes of that file (eg does it have a pkzip header?) and some CLI tools still do this (eg `file` does exactly that. `grep` reads the first 1000 (exact number escapes me) bytes and if there is a null byte (0x00) then it is assumed to by a binary file rather than text and outputs an error.
As an aside, one of the hobby file systems I wrote was long the kinds of lines of (2) too. It used vanilla MySQL/MariaDB as the back end because one of it's features was that you could then connect to a remote filesystem via a simple MySQL connection string. It was a pretty fun project and I'd gotten all the read operations working but there was a few bugs with the write operations that I never fully solved and I eventually lost interest when I starting working other projects.
> P.S. maybe a more productive direction: what is you recommended way for setting up some baloo or some other linux indexer for running mdfind-style commandline queries with it (I don't want KDE or Gnome and I think baloosearch vs mdfind is also easier to compare directly)?
Honestly I don't know. I might not like Windows much as a platform but I do really like the explorer.exe shell as UI paradigm. So I tend to gravitate towards KDE on Linux (plus I think the KDE team have done a great job refining that paradigm in ways the Microsoft have failed to). Krunner has always "just worked" for me so I haven't spent any energy looking for ways to replace it. However I'm sure there will be some guides online about setting up runners (are they called?) on Linux given the diversity of it's ecosystem.