Feels like saying Elasticsearch (and similar) tools are dead because we can just grep our way through things. I'd love to see more data on this.
The RAG Obituary: Killed by agents, buried by context windows
91–100 of 185 posts
Re: The RAG Obituary: Killed by agents, buried by context windows
#92This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…
But couldn’t an LLM search for documents in that enterprise knowledge base just like humans do, using the same kind of queries and the same underlying search infrastructure?
I wouldn't say humans are efficient at that so no reason to copy, other than as a starting point.
Re: The RAG Obituary: Killed by agents, buried by context windows
#93Earlier quoted context omitted.
Agentic retrieval is really more a form of deep research (from a product standpoint there is very little difference). The key is that LLMs > rerankers, at least when you're not at webscale where the cost differential is prohibitive.
LLMs > rerankers. Yes! I don't like rerankers. They are slow, the context window is small (4096 tokens), it's expensive... It's better when the LLM reads the whole file versus some top_chunks.
Rerankers are orders of magnitude faster and cheaper than LLMs. Typical latency out of the box on a decent sized cross encoder (~4B) will be under 50ms on cheap gpus like an A10G. You won’t be able to run a fancy LLM on that hardware and without tuning you’re looking at hundreds of ms minimum.
More importantly, it’s a lot easier to fine tune a reranker on behavior data than an LLM that makes dozens of irrelevant queries.
Re: The RAG Obituary: Killed by agents, buried by context windows
#94Agentic search with a handful of basic tools (drawn from BM25, semantic search, tags, SQL, knowledge graph, and a handful of custom retrieval functions) blows the lid off RAG in my experience. The downside is it takes longer. A single “investigation” can easily use 20-30 different function calls. RAG is like a static one-shot version of this and while the results are inferior the process is also a lot faster.
Hey, I’m interested in what you call “agentic search”. Did you roll your own or are you using a set of integrated tools?
I’ve used LightRAG and looking to integrate it with OpenWebUI and possibly air weave which was a show HN earlier.
My data is highly structured and has references between documents, so I wanted to leverage that structure for better retrieval and reasoning.
Re: The RAG Obituary: Killed by agents, buried by context windows
#95Earlier quoted context omitted.
> Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems Great point, but this grep in a loop probably falls apart (i.e. becomes non-performant) at 1000s of docs, not millions and 10s of simultaneous users
Why does grep in a loop fall apart? It’s expensive, sure, but LLM costs are trending toward zero. With Sonnet 4.5, we’ve seen models get better at parallelization and memory management (compacting conversations and highlighting findings).
"LLM costs are trending toward zero". They will never be zero for the cutting edge. One could argue that costs are zero now via local models but enterprises will always want the cutting edge which is likely to come with a cost
Re: The RAG Obituary: Killed by agents, buried by context windows
#96Earlier quoted context omitted.
But couldn’t an LLM search for documents in that enterprise knowledge base just like humans do, using the same kind of queries and the same underlying search infrastructure?
I wouldn't say humans are efficient at that so no reason to copy, other than as a starting point.
Maybe not efficient, but if the LLMs can't even reach this benchmark then I'm not sure.
Re: The RAG Obituary: Killed by agents, buried by context windows
#97This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…
Is letting an agent use grep not a form of RAG? I know usually RAG is done with vector databases but grep is definitely a form of retrieval, and it’s augmenting the generation.
Re: The RAG Obituary: Killed by agents, buried by context windows
#98Saying that RAG alone is complex and should be superseded by agentic search is a bit weak. Agentic search makes more sense when your pipeline becomes more complicated: RAG+MCP+Client calls, it is then when you can see that LLM starts behaving erratically and cannot answer the question well. You then want better control over streams of content and intents which could be solved by smaller agents looping over the data.
Re: The RAG Obituary: Killed by agents, buried by context windows
#99After building a few RAG based apps I was curious to try the ClaudeCode based approach that is mentioned by the author. So I built a python service that exposes ripgrep to a rest api: https://github.com/masterkram/jaguar
This makes it possible to quickly deploy this on coolify and quickly build an agent that can use ripgrep on any of your uploaded files.
Re: The RAG Obituary: Killed by agents, buried by context windows
#100This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…
But couldn’t an LLM search for documents in that enterprise knowledge base just like humans do, using the same kind of queries and the same underlying search infrastructure?
Yes but that would be worse than many RAG approaches, which were implemented precisely because there is no good way to cleanly search through a knowledge base for a million different reasons.
At that point, you are just doing Agentic RAG, or even just Query Review + RAG.
I mean, yeah, agentic RAG is the future. It's still RAG though.