Live data from Hacker News

The RAG Obituary: Killed by agents, buried by context windows

nicolasbustamante.com

101–110 of 185 posts

Re: The RAG Obituary: Killed by agents, buried by context windows

#101
I am so tired of these undifferentiated takes.

These types of articles regularly come from people who don't actually build SCALE systems with LLMs. Or, people who want to sell you on a new tech. And the frustrating thing is: They ain't even wrong.

Top-K RAG via vector search is not a sufficient solution. It never really was for most interesting use-cases.

Of course, take easiest and most structured - in a sense perfectly indexed - data (code repos) and claim that "RAG is dead". Again. Now try this with billions of unstructured tokens where the LLM really needs to do something with the entire context (like, confirm that something is NOT in the documents), where even the best LLM loses context coherence after like 64k tokens for complex tasks. Good luck!

The truth is: Whether its Agentic RAG, Graph RAG, or a combination of these with ye olde top-k RAG - it's still RAG. You are going to Retrieve, and then you are going to use a system of LLM agents to generate stuff with it. You may now be able to do the first step smarter. It's still Rag tho.

The latest Antrophic whoopsy showed that they also haven't solved the context rot issue. Yes you can get a 1M context scaled version of Claude, but then the small/detail scale performance is so garbage that misrouted customers loose their effin mind.

"My LLM is just gonna ripgrep through millions of technical doc pdfs identified only via undecipherable number-based filenames and inconsistent folder structures"

lol, and also, lmao

Re: The RAG Obituary: Killed by agents, buried by context windows

#102
People say “agents not RAG”, but one framing is that this describes RAG where the database is a file system and bash is the query language (with other cli tools installed it can use, including curl, jq, grep). With writing its own notes on the filesystem structure and maintaining them as a way to “index the database” It is still using code to selectively grab the chunks of data it needs rather than than putting everything in context. It’s just better RAG?

Re: The RAG Obituary: Killed by agents, buried by context windows

#104

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

I agree.

A great many pundits don't get, that RAG means: "a technique that enables large language models (LLMs) to retrieve and incorporate new information"

So, RAG is a pattern that is as a principle applied to almost every process. Context windows? Ok, I won't get into all the nitty gritty details here (embedded, small storage device, security, RAM defects, cost and storage of contexts for different contexts etc.), just a hint, that the act of filling a context is what? Applied RAG.

RAG is not a architecture, it is a principle. A structured approach. There is a reason, why nowadays many refer to RAG as search engine.

All we know about knowledge, there is only one entity with a infinite context window. We still call it God not cloud.

Re: The RAG Obituary: Killed by agents, buried by context windows

#105
I get the reasoning behind “letting the agent use grep in a loop“, after all, it is very similar to how humans would explore a document base with ctrl-f. But wouldn’t humans also use vector search all the time if it were as available as ctrl-f? So maybe not ditch vector search but provide it as a tool to the agent. Increased complexity aside, letting the agent explore a huge document base with “vector search in a loop“ should be more powerful that with grep in a loop. Overall I liked the article.

Re: The RAG Obituary: Killed by agents, buried by context windows

#106

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

I don't get it. Isn't grep RAG?

Re: The RAG Obituary: Killed by agents, buried by context windows

#107

I get the reasoning behind “letting the agent use grep in a loop“, after all, it is very similar to how humans would explore a document base with ctrl-f. But wouldn’t humans also use vector search all the time if it were as available as ctrl-f? So maybe not ditch vector search but provide it as a tool to the agent. Increased complexity aside, letting the agent explore a huge document base with “vector search in a loo…

My mental model (in the "all models are wrong, some are useful" sense) is that vector search is the thing that gives you the terms to grep for.

Re: The RAG Obituary: Killed by agents, buried by context windows

#108

Earlier quoted context omitted.

We always suspected find+grep+xargs was Turing-complete, and now Claude is proving it.

That's one of the most nonsensical comments on all of hackernews. A Markov change could have wrote it. What do you mean Turing complete? Obviously all 3 programs are running on a Turing complete machine. Xargs is a runner for other commands, obviously those commands can be Turing complete. I haven't heard of anybody working on a _proof_ for the Turing completeness of xargs, and I think the only conference willing to…

Can’t tell which of these two comments is the joke …

Re: The RAG Obituary: Killed by agents, buried by context windows

#109

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

Not to mention, unless you want to ship entire containers, you are beholden to the unknown quirks of tools on whatever system your agent happens to execute on. It's like taking something already nondeterministic and extremely risky and ceding even more control—let's all embrace chaos. Generative AI is here to stay, but I have a feeling we will look back on this period of time in software engineering as a sort of dark…

> It's pathetic that this industry so easily sold itself to the illogical sway of marketers and capital.

What are you implying. Capital always owned the industry except some really small coops and FOSS communities.

Re: The RAG Obituary: Killed by agents, buried by context windows

#110

I get the reasoning behind “letting the agent use grep in a loop“, after all, it is very similar to how humans would explore a document base with ctrl-f. But wouldn’t humans also use vector search all the time if it were as available as ctrl-f? So maybe not ditch vector search but provide it as a tool to the agent. Increased complexity aside, letting the agent explore a huge document base with “vector search in a loo…

My mental model (in the "all models are wrong, some are useful" sense) is that vector search is the thing that gives you the terms to grep for.

Or the thing that ranks the term based result. That’s the fun these days: it’s all whatever fits your problem.
Post reply on HN