[flagged]
We replaced RAG with a virtual filesystem for our AI documentation assistant
151–160 of 190 posts
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#152Earlier quoted context omitted.
I get what you’re saying, and you’re right, however I can also see where they’re coming from: Empirically, agents (especially the coding CLIs) seem to be doing so much better with files, even if the tooling around them is less than ideal. With other custom tools they instantly lose 50 IQ points, if they even bother using the tools in the first place.
Sorry, this still makes no sense. LLMs don't care about files. The way most codings systems work is that they simply provide the whole file to the LLM rather than a subset of it. That's just a choice in how you implemented your RAG search system and database. In this case the "record" is big, a file. No doubt that works for code, but it's nonsensical outside that. E.g. for wikipedia the logical unit would likely be a…
> LLMs don't care about files
They do. I highly suggest not try to derive LLMs' behaviors (in your mind) from first principles, but actually use them.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#153Let's say I want a free, local or free-tier-llm, simple solution to search information mostly from my emails and a little bit from text, doc and pdf files. Are there any tool I should try to have ollamma or gemini able to reply with my own knowledge base?
https://onyx.app/ This could be useful.
Are you using it? I will definitely give it a shot, any pointers to online resources will be appreciated
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#154Traditional RAG is a poor fit for this generation of LLMs, because it doesn't fit the "agentic tool use" workflow at all.
Self-guided "grep on a filesystem" often beats RAG because it allows the LLM to run "closed loop" and iteratively refine its queries until it obtains results. Self-guided search loop is a superset of what methods like reranking try to do.
I don't think vector search and retrieval is dead, but the old-fashioned RAG is. Vector search would have to be reengineered to fit into the new agentic workflows, so that the advantages of agentic LLMs can compound with that of vector search - because in current day "grep vs RAG" matchups, the former is already winning on the agentic merits.
"Optimize grep-centric search" is a surprisingly reasonable stopgap in the meanwhile.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#155The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…
I spent a while working on a retrieval system for LLMs and ended up reinventing a concordance (which is like an index).
It's basically the same thing as Google's inverted index, which is how Google search works.
Nothing new under the sun :)
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#156I think the interesting bit here is that filesystems give the model something it can actually operate on (ls, grep, etc), not just query.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#157[flagged]
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#158The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…
Aren’t most successful RAGs using a combination of embedding similarity + BM25 + reranking? I thought there were very few RAGs that only did pure embedding similarity, but I may be mistaken.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#159[dead]
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#160This is essentially tool use with a filesystem interface — the LLM decides what to read instead of a retrieval pipeline choosing for it. Clean idea, and it sidesteps the chunking problem entirely.
Curious about the latency though. RAG is one round trip: embed query, fetch chunks, generate. This approach seems like it needs multiple LLM calls to navigate the tree before it can answer. How many hops does it typically take, and did you have to do anything special to keep response times reasonable?