I am not familiar with the tech stack they use, but from an outsider point of view, I was sort of expecting some kind of fuse solution. Could someone explain why they went through a fake shell? There has to be a reason.
We replaced RAG with a virtual filesystem for our AI documentation assistant
21–30 of 190 posts
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#22I am not familiar with the tech stack they use, but from an outsider point of view, I was sort of expecting some kind of fuse solution. Could someone explain why they went through a fake shell? There has to be a reason.
100% agree a FUSE mount would be the way to go given more time and resources.
Putting Chroma behind a FUSE adapter was my initial thought when I was implementing this but it was way too slow.
I think we would also need to optimize grep even if we had a FUSE mount.
This was easier in our case, because we didn’t need a 100% POSIX compatibility for our read only docs use case because the agent used only a subset of bash commands anyway to traverse the docs. This also avoids any extra infra overhead or maintenance of EC2 nodes/sandboxes that the agent would have to use.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#23I think generally we are going from vector based search, to agentic tool use, and hierarchy based systems like skills.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#24haha, sweet. One of the cooler things I've read lately
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#25The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…
I think it's cool that LLMs can effectively do this kind of categorization on the fly at relatively large scale. When you give the LLM tools beyond just "search", it really is effectively cheating.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#26I think generally we are going from vector based search, to agentic tool use, and hierarchy based systems like skills.
Vector search has moved from a "complete solution" to just one tool among many which you should likely provide to an agent.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#27This puts a lot of LLM in front of the information discovery. That would require far more sophisticated prompting and guardrails. I'd be curious to see how people architect an LLM->document approach with tool calling, rather than RAG->reranker->LLM. I'm also curious what the response times are like since it's more variable.
Hmmm, the post is an attempt to explain that Mintlify migrated from embedding-retrieval->reranker->LLM to an agent loop with access to call POSIX tools as it desires. Perhaps we didn't provide enough detail?
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#28I think this is a great approach for a startup like Mintlify. I do have skepticism around how practical this would be in some of the “messier” organisations where RAG stands to add the most value. From personal experience, getting RAG to work well in places where the structure of the organisation and the information contained therein is far from hierarchical or partition-able is a very hard task.
Modern OCR tooling is quite good. If the knowledge you are adding into your search database is able to be OCR'd then I think the approach we took here is able to be generalized.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#29This is definitely the way. There are good use cases for real sandboxes (if your agent is executing arbitrary code, you better it do so in an air-gapped environment). But the idea of spinning up a whole VM to use unix IO primitives is way overkill. Makes way more sense to let the agent spit our unix-like tool calls and then use whatever your prod stack uses to do IO.
100% agree. However, if there were no resource tradeoffs, then a FUSE mount would probably be the way to go.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#30If grep and ls do the trick, then sure you don't need RAG/embeddings. But you also don't need an LLM: a full text search in a database will be a lot more performant, faster and use less resources.