Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced RAG with a virtual filesystem for our AI documentation assistant

mintlify.com

Post reply on HN
Print view View original

41–50 of 190 posts

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#41

Post by skeptrune »

HanClinto wrote: post #37

> "The agent doesn't need a real filesystem; it just needs the illusion of one. Our documentation was already indexed, chunked, and stored in a Chroma database to power our search, so we built ChromaFs: a virtual filesystem that intercepts UNIX commands and translates them into queries against that same database. Session creation dropped from ~46 seconds to ~100 milliseconds, and since ChromaFs reuses infrastructure…

We would also be super interested to see that comparison. I agree that there isn't a specific reason why Chroma would be required to build something like this.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#42

Post by dominotw »

kenforthewin wrote: post #32

I don't get it - everybody in this thread is talking about the death of vector DBs and files being all you need. The article clearly states that this is a layer on top of their existing Chroma db.

what value is chromadb adding in that setup

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#43

Post by ghywertelling »

tschellenbach wrote: post #23

I think generally we are going from vector based search, to agentic tool use, and hierarchy based systems like skills.

Agents doing retrieval has been around for quite a while

https://huggingface.co/docs/smolagents/en/examples/rag

Agentic RAG: A More Powerful Approach We can overcome these limitations by implementing an Agentic RAG system - essentially an agent equipped with retrieval capabilities. This approach transforms RAG from a rigid pipeline into an interactive, reasoning-driven process.

The innovation of the blogpost is in the retrieval step.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#44

Post by czhu12 »

softwaredoug wrote: post #2

The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…

Similar effort with PageIndex [1], which basically creates a table of contents like tree. Then an LLM traverses the tree to figure out which chunks are relevant for the context in the prompt.

1: https://github.com/VectifyAI/PageIndex

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#45

Post by ghywertelling »

maille wrote: post #7

Let's say I want a free, local or free-tier-llm, simple solution to search information mostly from my emails and a little bit from text, doc and pdf files. Are there any tool I should try to have ollamma or gemini able to reply with my own knowledge base?

https://onyx.app/

This could be useful.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#46

Post by rafterydj »

softwaredoug wrote: post #13
wielebny wrote: post #5

Earlier quoted context omitted.

Someone simply assumed at some point that RAG must be based on vector search, and everyone followed.

It’s something of a historical accident We started with LLMs when everyone in search was building question answering systems. Those architectures look like the vector DB + chunking we associate with RAG. Agents ability to call tools, using any retrieval backend, call that into question. We really shouldn’t start RAG with the assumption we need that. I’ll be speaking about the subject in a few weeks https://maven.com/…

Stuck it on my calendar, looking forward to it.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#48

Post by skeptrune »

bluegatty wrote: post #33

RAG should no have have been represented as a context tool but rather just vector querying ad an variation of search/query - and that's it. We were bitten by our own nomenclature. Just a small variation in chosen acronym ... may have wrought a different outcome. Different ways to find context are welcome, we have a long way to go!

agreed!

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#49

Post by Galanwe »

skeptrune wrote: post #22
Galanwe wrote: post #21

I am not familiar with the tech stack they use, but from an outsider point of view, I was sort of expecting some kind of fuse solution. Could someone explain why they went through a fake shell? There has to be a reason.

100% agree a FUSE mount would be the way to go given more time and resources. Putting Chroma behind a FUSE adapter was my initial thought when I was implementing this but it was way too slow. I think we would also need to optimize grep even if we had a FUSE mount. This was easier in our case, because we didn’t need a 100% POSIX compatibility for our read only docs use case because the agent used only a subset of bash…

Makes sense, thanks for clarifying!

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#50

Post by oceansky »

TeMPOraL wrote: post #20
softwaredoug wrote: post #13

Earlier quoted context omitted.

It’s something of a historical accident We started with LLMs when everyone in search was building question answering systems. Those architectures look like the vector DB + chunking we associate with RAG. Agents ability to call tools, using any retrieval backend, call that into question. We really shouldn’t start RAG with the assumption we need that. I’ll be speaking about the subject in a few weeks https://maven.com/…

Right. R in RAG stands for retrieval , and for a brief moment initially, it meant just that: any kind of tool call that retrieves information based on query, whether that was web search, or RDBMS query, or grep call, or asking someone to look up an address in a phone book. Nothing in RAG implies vector search and text embeddings (beyond those in the LLM itself), yet somehow people married the acronym to one very part…

I'm still using the old definition, never got the memo.
Post reply on HN