Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAG Obituary: Killed by agents, buried by context windows

nicolasbustamante.com

Post reply on HN
Print view View original

121–130 of 185 posts

Re: The RAG Obituary: Killed by agents, buried by context windows

#121

Post by lossolo »

gk1 wrote: post #120
lossolo wrote: post #117

Earlier quoted context omitted.

In RAG, you operate on embeddings and perform vector search, so if you search for fat lady, it might also retrieve text like huge queen, because they're semantically similar. Grep on the other hand, only matches exact strings, so it would not find it.

R in RAG is for retrieval… of any kind. It doesn’t have to be vector search.

Sure, but vector search is the dominant form of RAG, the rest are niche. Saying "RAG doesn’t have to use vectors" is like saying "LLMs don't have to use transformers". Technically true, but irrelevant when 99% of what's in use today does.

Re: The RAG Obituary: Killed by agents, buried by context windows

#122

Post by EdwardDiego »

delusional wrote: post #83
Analemma_ wrote: post #34

Earlier quoted context omitted.

We always suspected find+grep+xargs was Turing-complete, and now Claude is proving it.

That's one of the most nonsensical comments on all of hackernews. A Markov change could have wrote it. What do you mean Turing complete? Obviously all 3 programs are running on a Turing complete machine. Xargs is a runner for other commands, obviously those commands can be Turing complete. I haven't heard of anybody working on a _proof_ for the Turing completeness of xargs, and I think the only conference willing to…

It was a joke.

Re: The RAG Obituary: Killed by agents, buried by context windows

#123

Post by grafmax »

davidmckayv wrote: post #42

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

RAG doesn’t just mean word vectors but can include keyword search. Claude using grep is a form of RAG.

Re: The RAG Obituary: Killed by agents, buried by context windows

#124

Post by cantor_S_drug »

devmor wrote: post #87

This reads like someone AI-generated prose to defend something they want to invest in and decry something it competes with. It does not come off as honest, written by a human, or useful to anyone outside of the specific, narrow contexts the "author" sees for the technologies mentioned. Frankly, reading through this at makes me feel as though I am a business analyst or engineering manager being presented with a projec…

How come this isn't the top comment? This post screams AI.

Re: The RAG Obituary: Killed by agents, buried by context windows

#125

Post by edanm »

grafmax wrote: post #123
davidmckayv wrote: post #42

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

RAG doesn’t just mean word vectors but can include keyword search. Claude using grep is a form of RAG.

In practice this is not how the term is used.

It bugs me, because the acronym should encompass any form of retrieval - but in practice, people use RAG to specifically refer to embedding-vector-lookups, hence it making sense to say that it's "dying" now that other forms of retrieval are better.

Re: The RAG Obituary: Killed by agents, buried by context windows

#126

Post by tw1984 »

CuriouslyC wrote: post #43

RAG isn't dead, RAG is just fiddly, you need to tune retrieval to the task. Also, grep is a form of RAG, it just doesn't use embeddings.

No, grep is not RAG. RAG is all about embeddings + vector search + LLM working under a fixed workflow.

Saying grep is also RAG is like saying ext4 + grep is a database.

Re: The RAG Obituary: Killed by agents, buried by context windows

#127

Post by jgalt212 »

nbstme wrote: post #54
jgalt212 wrote: post #51

Earlier quoted context omitted.

> Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems Great point, but this grep in a loop probably falls apart (i.e. becomes non-performant) at 1000s of docs, not millions and 10s of simultaneous users

Why does grep in a loop fall apart? It’s expensive, sure, but LLM costs are trending toward zero. With Sonnet 4.5, we’ve seen models get better at parallelization and memory management (compacting conversations and highlighting findings).

If LLM costs are trending towards zero, please explain the $600B openai when Oracle and the $100B deal with Nvidia.

And if you think those deals are bogus, like I do, you still need to explain surging electricity prices.

Re: The RAG Obituary: Killed by agents, buried by context windows

#128

Post by jgalt212 »

dangoodmanUT wrote: post #78
queenkjuul wrote: post #71

Earlier quoted context omitted.

I don't see how it can trend to zero when none of the vendors are profitable. Uber and doordash et. al. increased in price over time. The era of "free" LLM usage can't be permanent

Google’s inference is profitable

Not on the SERP page. The zero click Internet is bad for content producers and for those who sell ads (Google).

Re: The RAG Obituary: Killed by agents, buried by context windows

#129

Post by guipsp »

lukaslalinsky wrote: post #113
eru wrote: post #26

Earlier quoted context omitted.

That's what I used to use as a human, but then I finally overcame my laziness in setting up integration between my editor and compiler (and similar) and got 'jump to definition' working. (Well, I didn't overcome my laziness directly. I just switched from being lazy and not setting up vim and Emacs with the integrations, to trying out vscode where this was trivial or already built in.)

Do you trust 'jump to definition'. Obviously it depends on the language server, but it's best effort. I'm often frustrated when it doesn't work, because I broke the code in some way. Or it jumps to a specific definition, but there are multiple. If I was as quick at opening and reading files as claude code, I'd prefer grep with context around the searched term.

In java, for example, jump to definition is pretty flawless.

Re: The RAG Obituary: Killed by agents, buried by context windows

#130

Post by lemonlearnings »

davidmckayv wrote: post #42

This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap. Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use…

Isn't grep + LLM a form of RAG anyway?
Post reply on HN