I'm working on a filesystem for agents for similar reasons - https://clawfs.dev - lmk if your team would like an invite..
We replaced RAG with a virtual filesystem for our AI documentation assistant
161–170 of 190 posts
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#162[dead]
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#163Earlier quoted context omitted.
[flagged]
There a lot of methods in IR/RAG that maintain structure as metadata used in a hybrid fusion to augment search. Graph databases is an extreme form but some RAG pipelines pull out and embed the metadata with the chunk together. In the specific case of code, other layered approaches like ColGrep (late interaction) show promise.... the point is most search most of the time will benefit from a combination approach not a…
Just like the approach in the article.
Everything is based on the metadata stored with chunks, just allowing the agent to navigate that metadata through ls, cd, find and grep.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#164But SQLite is notoriously 35% faster than the filesystem [0], so why not use that? [0] https://news.ycombinator.com/item?id=14550060
SQLite + GPT5.4 works very well for me. My biggest success is a Roslyn method that takes a .NET solution and converts it into a SQLite database with Files, Lines, Symbols, and References tables. I've found this approach to perform substantially better than a flat, file-based setup (i.e., like what Copilot provides in Visual Studio). Especially, for very large projects. 100+ megs of source is no problem. The relationa…
Is this something you can share in more detail? Did you document a skill for the LLM to use? And with what tasks do you see most improvement?
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#165This is one of the most confusing claims I've seen in a long time. Grep and others over files would be the equivalent of an old fashioned keyword search where most RAG uses vector search. But everything else they claim about a file system just suggests that they don't know anything about databases. I'm not familiar with how most out of the box RAG systems categorize data, but with a database you can index content lit…
Isn’t this the approach described in the article?
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#166This is essentially tool use with a filesystem interface — the LLM decides what to read instead of a retrieval pipeline choosing for it. Clean idea, and it sidesteps the chunking problem entirely. Curious about the latency though. RAG is one round trip: embed query, fetch chunks, generate. This approach seems like it needs multiple LLM calls to navigate the tree before it can answer. How many hops does it typically t…
In their case it was competing with cloning an entire repo before starting a session which was taking 10s of seconds.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#167I don't get it - everybody in this thread is talking about the death of vector DBs and files being all you need. The article clearly states that this is a layer on top of their existing Chroma db.
I assumed they had other use cases where vector search was required.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#168We use both a virtual file system and RAG — they each excel in different areas. The trick with RAG is the quality of data: we use an LLM to chunk into semantically cohesive sections, as well as generate metadata (including fact triples and links to other related chunks in the document) for every chunk as well as the document as a whole. We use voyage contextual embeddings to then embed each chunk with the document and chunk metadata. Works incredibly well. At retrieval time the agent can follow chunk links if needed, as well as analyze the raw file in the vfs. High quality instruction based reranking helps a lot too! We are often looking over 10s of thousands of documents and it’d be very inefficient to have our agents analyze just the vfs without rag.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#169We use both a virtual file system and RAG — they each excel in different areas. The trick with RAG is the quality of data: we use an LLM to chunk into semantically cohesive sections, as well as generate metadata (including fact triples and links to other related chunks in the document) for every chunk as well as the document as a whole. We use voyage contextual embeddings to then embed each chunk with the document an…
Our vfs is also pretty powerful too, though: it is all backed by postgres then projected into files/directories for our agents. They get basic grep etc but also optimized fts tools for bm25, jq, and preview tools that show representative slices of large documents. All on top of Pydantic AI.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#170> The obvious way to do this is to just give the agent a real filesystem. Most harnesses solve this by spinning up an isolated sandbox and cloning the repo. We already use sandboxes for asynchronous background agents where latency is an afterthought, but for a frontend assistant where a user is staring at a loading spinner, the approach falls apart. Our p90 session creation time (including GitHub clone and other setup) was ~46 seconds.
Am I the only one who read this and thought this is fucking insane? Who in their right mind would even consider spinning up a virtual machine and cloning a repo on every search query? And if all you need is a real filesystem why would you emulate a filesystem on top of a database (Chroma)? If you need a filesystem just use an actual filesystem! This sounds like insane gymnastics just to fit a “serverless” workflow. 850,000 searches a month (less than 1 request per second) sounds like something a single raspberry pi or Mac Mini could handle.