Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced RAG with a virtual filesystem for our AI documentation assistant

mintlify.com

Post reply on HN
Print view View original

91–100 of 190 posts

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#91

Post by darkteflon »

skeptrune wrote: post #22
Galanwe wrote: post #21

I am not familiar with the tech stack they use, but from an outsider point of view, I was sort of expecting some kind of fuse solution. Could someone explain why they went through a fake shell? There has to be a reason.

100% agree a FUSE mount would be the way to go given more time and resources. Putting Chroma behind a FUSE adapter was my initial thought when I was implementing this but it was way too slow. I think we would also need to optimize grep even if we had a FUSE mount. This was easier in our case, because we didn’t need a 100% POSIX compatibility for our read only docs use case because the agent used only a subset of bash…

Did you guys look at Firecracker-based options such as E2B and Fly.io? We’ve had positive early results on latency, but yeah … too early to tell where we end up on cost.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#92

Post by skeptrune »

darkteflon wrote: post #91
skeptrune wrote: post #22

Earlier quoted context omitted.

100% agree a FUSE mount would be the way to go given more time and resources. Putting Chroma behind a FUSE adapter was my initial thought when I was implementing this but it was way too slow. I think we would also need to optimize grep even if we had a FUSE mount. This was easier in our case, because we didn’t need a 100% POSIX compatibility for our read only docs use case because the agent used only a subset of bash…

Did you guys look at Firecracker-based options such as E2B and Fly.io? We’ve had positive early results on latency, but yeah … too early to tell where we end up on cost.

Yea we did and actually use Daytona for another product, but it would have been too slow here.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#93

Post by skeptrune »

readitalready wrote: post #56
skeptrune wrote: post #22

Earlier quoted context omitted.

100% agree a FUSE mount would be the way to go given more time and resources. Putting Chroma behind a FUSE adapter was my initial thought when I was implementing this but it was way too slow. I think we would also need to optimize grep even if we had a FUSE mount. This was easier in our case, because we didn’t need a 100% POSIX compatibility for our read only docs use case because the agent used only a subset of bash…

Yah my Claude Code agents run a ton of Python and bash scripts. You're probably missing out on a lot of tool use cases without full tool use through POSIX compatibility.

agreed. hopefully we can get there soon

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#94

Post by skeptrune »

kenforthewin wrote: post #54
skeptrune wrote: post #51

Earlier quoted context omitted.

yea chromadb is not the point. multiple data storage solutions work

I see .. so you're not using the vectors at all. Where are the evaluations showing this chromaFS approach is performing better than vectors?

Working on publishing those, but publishing benchmarks requires a lot of attention to detail so it will likely be a bit longer.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#95

Post by all2 »

mandeepj wrote: post #14

> even a minimal setup (1 vCPU, 2 GiB RAM, 5-minute session lifetime) would put us north of $70,000 a year based on Daytona's per-second sandbox pricing ($0.0504/h per vCPU, $0.0162/h per GiB RAM) $70k? how about if we round off one zero? Give us $7000. That number still seems to be very high.

At that point I would buy an old mini PC off of ebay and just put it on my desk.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#97

Post by brap »

tensor wrote: post #80
brap wrote: post #77

Earlier quoted context omitted.

I get what you’re saying, and you’re right, however I can also see where they’re coming from: Empirically, agents (especially the coding CLIs) seem to be doing so much better with files, even if the tooling around them is less than ideal. With other custom tools they instantly lose 50 IQ points, if they even bother using the tools in the first place.

Sorry, this still makes no sense. LLMs don't care about files. The way most codings systems work is that they simply provide the whole file to the LLM rather than a subset of it. That's just a choice in how you implemented your RAG search system and database. In this case the "record" is big, a file. No doubt that works for code, but it's nonsensical outside that. E.g. for wikipedia the logical unit would likely be a…

>LLMs don't care about files.

Oh but they do. These CLI agents are trained and specifically tuned to work with the filesystem. It’s not about the content or how it’s actually stored, it’s about the familiar access patterns.

I can’t begin to tell you how many times I’ve seen a coding agent figure out it can get some data directly from the filesystem instead of a dedicated, optimized tool it was specifically instructed to use for this purpose.

You basically can’t stop these things from messing with files, it’s in their DNA. You block one shell command, they’ll find another. Either revoke shell access completely or play whackamole. You cannot believe how badly they want to work with files.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#98

Post by neuzhou »

softwaredoug wrote: post #2

The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…

[flagged]
Post reply on HN