Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAG Obituary: Killed by agents, buried by context windows

nicolasbustamante.com

Post reply on HN
Print view View original

151–160 of 185 posts

Re: The RAG Obituary: Killed by agents, buried by context windows

#151

Post by athrowaway3z »

I had seen RAG mentioned a lot before I had gotten into LLM agents. I assumed it was tricky and required real deep model training knowledge.

My first real AI use (beyond copy-paste ChatGPT) was Claude Code. I figured out in a few days to just write scripts and CLAUDE.md how to use them. For instance, one that prints comments and function names in a file is a few lines of python. MCP seemed like context bloat when a `tools/my-script -h` would put it in context on request.

Eventually stumbled on some more RAG a few weeks later, so decided to read up on it and... what? That's it? A 'prelude function' to dump 'probably related' things into the context?

It seems so obviously the wrong way to go from my perspective, so am I missing something here?

Re: The RAG Obituary: Killed by agents, buried by context windows

#153

Post by torginus »

davidmckayv wrote: post #42

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

Yeah, 'RAG' is quite literal tool use, where the tool is a vector search engine more or less.

What was described as 'RAG' a year ago now is a 'knowledge search in vector db MCP', with the actual tool and mechanism of knowledge retrieval being the exact same.

Re: The RAG Obituary: Killed by agents, buried by context windows

#154

Post by malshe »

> Table Integrity: Financial tables are never split—income statements, balance sheets, and cash flow statements remain atomic units with headers and data together

In 10-k and 10-q often there are no table headers. This is particularly true for the consolidated notes to financial statements section. Standalone tables could be pretty much meaningless because you won't even know what they are reporting. For example, a table that simply mentions terms like beginning balance and ending balance can be reporting inventory, warranty, or short term debt. But the table does not mention these metrics at all and there are no headers. So I am curious to know how Fintool uses standalone tables. Do you retain text surrounding the tables in the same chunk as the table?

Re: The RAG Obituary: Killed by agents, buried by context windows

#155

Post by esafak »

This is just wrong. As many here have said, grep is RAG; just the most primitive kind. It means you miss typos, synonyms, semantic matches (e.g., "the payment service"), and AST matches. I have to deal with this when I use grep-based agents by handholding them and overpaying. grep is just something that enabled CLI-based tools to get to market faster. grep's dominance will fade as the landscape matures. The current pattern seems to be to outsource RAG to an MCP.

Re: The RAG Obituary: Killed by agents, buried by context windows

#157

Post by closeparen »

davidmckayv wrote: post #42

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

Cursor’s use of grep is bad. It finds definitions way slower and less accurately than I do using IDE indexing, which is frustratingly “right there.” Crazy that there’s not even LSP support in there.

Claude Code is better, but still frustrating.

Re: The RAG Obituary: Killed by agents, buried by context windows

#158

Post by CuriouslyC »

tw1984 wrote: post #126
CuriouslyC wrote: post #43

RAG isn't dead, RAG is just fiddly, you need to tune retrieval to the task. Also, grep is a form of RAG, it just doesn't use embeddings.

No, grep is not RAG. RAG is all about embeddings + vector search + LLM working under a fixed workflow. Saying grep is also RAG is like saying ext4 + grep is a database.

So you're saying grep isn't a form of information retrieval?

Re: The RAG Obituary: Killed by agents, buried by context windows

#159

Post by flyinglizard »

nbstme wrote: post #54
jgalt212 wrote: post #51

Earlier quoted context omitted.

> Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems Great point, but this grep in a loop probably falls apart (i.e. becomes non-performant) at 1000s of docs, not millions and 10s of simultaneous users

Why does grep in a loop fall apart? It’s expensive, sure, but LLM costs are trending toward zero. With Sonnet 4.5, we’ve seen models get better at parallelization and memory management (compacting conversations and highlighting findings).

They're not trending toward zero; they're just aggressively subsidized with oil money.

Re: The RAG Obituary: Killed by agents, buried by context windows

#160

Post by rohansood15 »

I don't get why folks are so dismissive here.

If you ever saw Claude Code/Codex use grep, you will find that it constructs complex queries that encompass a whole range of keywords which may not even be present in the original user query. So the 'semantic meaning' isn't actually lost.

And nobody is putting an entire enterprise's knowledge base inside the context window. How many enterprise tasks are there that need referencing more that a dozen docs? And even those that do, can be broken down into sub-tasks of manageable size.

Lastly, nobody here mentions how much of a pain it is to build, maintain and secure an enterprise vector database. People spend months cleaning the data, chunking and vectorizing it, only for newer versions of the same data making it redundant overnight. And good look recreating your entire permissioning and access control stack on top of the vector database you just created.

The RAG obituary is a bit provocative, and maybe that's intentional. But it's surprising how negative/dismissive the reactions in this thread are.

Post reply on HN