We replaced RAG with a virtual filesystem for our AI documentation assistant • Hacker News Board

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#131

Post by safety1st » Sat, Apr 04, 2026, 4:45 AM UTC

bartread wrote: post #130

safety1st wrote: post #129

Earlier quoted context omitted.

We were given a demo of a vector based approach, and it didn't work. They said our docs were too big and for some reason their chunking process was failing. So we ended up using a good old fashioned Elastic backend because that's what we know, and simply forwarding a few of these giant documents to the LLM verbatim along with the user's question. The results have been great, not a single complaint about accuracy, res…

> They said our docs were too big and for some reason their chunking process was failing. Why would the size of your docs have any bearing on whether or not the chunking process works? That makes no sense. Unless of course they're operating on the document entirely in memory which seems not very bright unless you're very confident of the maximum size of document you're going to be dealing with. (I implemented a RAG p…

I agree it makes no sense. The whole point of chunking is to handle large documents. If your chunking system fails because a document is too big, that seems like a pretty glaring omission. I just chalked it up to the tech being new and novel and therefore having more bugs/people not fully understanding how it worked/etc. It was a vendor and they never gave us more details.

Not all problems have to be solved. We just fell back to using older, more proven technology, started with the simplest implementation and iterated as needed, and the result was great.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#132

Post by adamsilvacons » Sat, Apr 04, 2026, 5:49 AM UTC

[flagged]

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#133

Post by nithril » Sat, Apr 04, 2026, 6:28 AM UTC

The news headline is misleading, it does not reflect the actual article title, which is much closer to what they truly did.

They did not replaced RAG because they are still using chunk and embedding. What they changed is the interface.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#134

Post by [deleted] » Sat, Apr 04, 2026, 6:51 AM UTC

bob1029 wrote: post #101

jdthedisciple wrote: post #57

But SQLite is notoriously 35% faster than the filesystem [0], so why not use that? [0] https://news.ycombinator.com/item?id=14550060

SQLite + GPT5.4 works very well for me. My biggest success is a Roslyn method that takes a .NET solution and converts it into a SQLite database with Files, Lines, Symbols, and References tables. I've found this approach to perform substantially better than a flat, file-based setup (i.e., like what Copilot provides in Visual Studio). Especially, for very large projects. 100+ megs of source is no problem. The relationa…

[deleted]

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#135

Post by jiusanzhou » Sat, Apr 04, 2026, 7:01 AM UTC

Clever use of just-bash to avoid the sandbox cold-start problem. The key insight here is that agents don't need a real filesystem — they need a familiar interface backed by whatever storage you already have. We're seeing the same pattern in coding agents: directory hierarchy turns out to be a surprisingly effective knowledge graph that LLMs navigate better than embedding-based retrieval, mostly because they've been heavily trained on shell interactions.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#136

Post by emson » Sat, Apr 04, 2026, 7:14 AM UTC

This is interesting as there is definitely a middle ground for agent memory. On the openclaw side you have a single MEMORY.md file on the other you have RAG and GraphRAG. I wonder if Agent memory should be more nuanced? When an agent learns something how should it promote or degrade these memory blocks - you don’t want a trading agent memorising a bad trading pattern, for example. Also the agent might want to recall semantically similar memories, but it might also want to retrieve block relationships or groups of blocks for different purposes. We’ve been exploring all these concepts with “elfmem” (sELF improving MEMory): https://github.com/emson/elfmem Would love your feedback!

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#137

Post by pertymcpert » Sat, Apr 04, 2026, 7:20 AM UTC

neuzhou wrote: post #98

softwaredoug wrote: post #2

The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain. We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents. https://softwaredoug.com/blog/2026/01/08/seman…

[flagged]

[flagged]

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#138

Post by pertymcpert » Sat, Apr 04, 2026, 7:22 AM UTC

tensor wrote: post #80

brap wrote: post #77

Earlier quoted context omitted.

I get what you’re saying, and you’re right, however I can also see where they’re coming from: Empirically, agents (especially the coding CLIs) seem to be doing so much better with files, even if the tooling around them is less than ideal. With other custom tools they instantly lose 50 IQ points, if they even bother using the tools in the first place.

Sorry, this still makes no sense. LLMs don't care about files. The way most codings systems work is that they simply provide the whole file to the LLM rather than a subset of it. That's just a choice in how you implemented your RAG search system and database. In this case the "record" is big, a file. No doubt that works for code, but it's nonsensical outside that. E.g. for wikipedia the logical unit would likely be a…

They do care about files. They also care about how you express yourself, your tone, all sorts of seemingly unimportant details.

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#139

Post by dfordp11 » Sat, Apr 04, 2026, 7:45 AM UTC

[dead]

Re: We replaced RAG with a virtual filesystem for our AI documentation assistant

#140

Post by lelanthran » Sat, Apr 04, 2026, 7:49 AM UTC

senordevnyc wrote: post #126

KPGv2 wrote: post #55

Earlier quoted context omitted.

You seem like someone who knows what they're doing, and I understand the theoretical underpinnings of LLMs (math background), but I have little kids that were born in 2016 and so the entire AI thing has left me in the dust. Never any time to even experiment. I am active in fandoms and want to create a search where someone can ask "what was that fanfic where XYZ happened?" and get an answer back in the form of links t…

Honestly, just from this question, I think you know enough that I’d go spend $20/month for a subscription to Codex, Claude Code, or Cursor, and ask them to teach you all this. I bet if you put in your comment verbatim with Opus 4.6 and went back and forth a bit, it could help you figure out exactly what you need and build a first version in a couple hours. Seriously, if you know the fundamentals and can poke and prod…

> Honestly, just from this question, I think you know enough that I’d go spend $20/month for a subscription to Codex, Claude Code, or Cursor, and ask them to teach you all this.

Paying $20/m sounds like overkill. I have tabs open for all of the most well-known AI chatbots. Despite trying my hardest, it is not possible to exhaust your free options just by learning.

Hell, just on the chatbots alone, small projects can be vibe-coded too! No $20/m necessary.