I dont understand the additional complexity of mocking bash when they could just provide grep, ls, find, etc tools to the LLM
We replaced RAG with a virtual filesystem for our AI documentation assistant
31–40 of 190 posts
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#32I don't get it - everybody in this thread is talking about the death of vector DBs and files being all you need. The article clearly states that this is a layer on top of their existing Chroma db.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#33RAG should no have have been represented as a context tool but rather just vector querying ad an variation of search/query - and that's it.
We were bitten by our own nomenclature.
Just a small variation in chosen acronym ... may have wrought a different outcome.
Different ways to find context are welcome, we have a long way to go!
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#34The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…
Someone simply assumed at some point that RAG must be based on vector search, and everyone followed.
It was the terminology that did that more than anything. The term 'RAG' just has a lot of consequential baggage. Unfortunately.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#35> even a minimal setup (1 vCPU, 2 GiB RAM, 5-minute session lifetime) would put us north of $70,000 a year based on Daytona's per-second sandbox pricing ($0.0504/h per vCPU, $0.0162/h per GiB RAM) $70k? how about if we round off one zero? Give us $7000. That number still seems to be very high.
Hm. I think a dedicated 16-core box with 64 ram can be had for under $1000/year.
It being dedicated there are no limits on session lifetime and it'd run 16 those sessions no problem, so the real price should be around ~$70/year for that load.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#36This puts a lot of LLM in front of the information discovery. That would require far more sophisticated prompting and guardrails. I'd be curious to see how people architect an LLM->document approach with tool calling, rather than RAG->reranker->LLM. I'm also curious what the response times are like since it's more variable.
Hmmm, the post is an attempt to explain that Mintlify migrated from embedding-retrieval->reranker->LLM to an agent loop with access to call POSIX tools as it desires. Perhaps we didn't provide enough detail?
That matches what I'm curious about. Where an LLM is doing the bulk of information discovery and tool calling directly. Most simpler RAGs have an LLM on the frontend mostly just doing simpler query clean up, subqueries and taxonomy, then again later to rerank and parse the data. So I'd imagine the prompting and guardrails part is much more complicated in an agent loop approach, since it's more powerful and open ended.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#37> "The agent doesn't need a real filesystem; it just needs the illusion of one. Our documentation was already indexed, chunked, and stored in a Chroma database to power our search, so we built ChromaFs: a virtual filesystem that intercepts UNIX commands and translates them into queries against that same database. Session creation dropped from ~46 seconds to ~100 milliseconds, and since ChromaFs reuses infrastructure we already pay for, the marginal per-conversation compute cost is zero."
Not to be "that guy" [0], but (especially for users who aren't already in ChromaDB) -- how would this be different for us from using a RAM disk?
> "ChromaFs is built on just-bash ... a TypeScript reimplementation of bash that supports grep, cat, ls, find, and cd. just-bash exposes a pluggable IFileSystem interface, so it handles all the parsing, piping, and flag logic while ChromaFs translates every underlying filesystem call into a Chroma query."
It sounds like the expected use-case is that agents would interact with the data via standard CLI tools (grep, cat, ls, find, etc), and there is nothing Chroma-specific in the final implementation (? Do I have that right?).
The author compares the speeds against the Chroma implementation vs. a physical HDD, but I wonder how the benchmark would compare against a Ramdisk with the same information / queries?
I'm very willing to believe that Chroma would still be faster / better for X/Y/Z reason, but I would be interested in seeing it compared, since for many people who already have their data in a hierarchical tree view, I bet there could be some massive speedups by mounting the memory directories in RAM instead of HDD.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#38Is this related to that thing where somehow the entire damn world forgot about the power of boolean (and other precise) searching?
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#39I dont understand the additional complexity of mocking bash when they could just provide grep, ls, find, etc tools to the LLM
I agree that would have been the way to go given more time and resources. However, setting up a FUSE mount would have taken significantly longer and required additional infrastructure.
Re: We replaced RAG with a virtual filesystem for our AI documentation assistant
#40I dont understand the additional complexity of mocking bash when they could just provide grep, ls, find, etc tools to the LLM
agents are trained on bash grep/ls/find, not on tool-calling grep/ls/find