Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pruning RAG context down to what the answer actually needs

kapa.ai

Post reply on HN
Print view View original

41–50 of 54 posts

Re: Pruning RAG context down to what the answer actually needs

#41

Post by kordlessagain »

seanhunter wrote: post #35
agentdev001 wrote: post #4

Am I wrong to be somewhat peeved by the use of "RAG" in these contexts? I always read things like this, and wonder if instead the author should be saying "Semantic Retrieval" or something something Vector, etc. Retrieval augmented generation captures tool-use, and; semantic search of course is really just a tool under the hood. To make an anology, in my mind, this is akin to saying "fuel air mixture system" when refe…

Yes you are wrong. RAG means retrieval-augmented generation. If you’re generating something and that generation is augmented by some retrieval you’re doing RAG. The retrieval doesn’t need to be from a vector db or even based on semantic similarity.

[flagged]

Re: Pruning RAG context down to what the answer actually needs

#42

Post by fuck_google »

daveKoala wrote: post #32
grewil2 wrote: post #26

"Three knobs matter:" I can't help reading articles with the radar on for signs of AI-generation nowadays. I have noticed that Claude sometimes uses the word knob for parameter, so here I get suspicious.

I am being constantly mocked for using there term 'knob fiddling', but I am a British child of the 70's :-)

[dead]

Re: Pruning RAG context down to what the answer actually needs

#43

Post by seanhunter »

kordlessagain wrote: post #41
seanhunter wrote: post #35

Earlier quoted context omitted.

Yes you are wrong. RAG means retrieval-augmented generation. If you’re generating something and that generation is augmented by some retrieval you’re doing RAG. The retrieval doesn’t need to be from a vector db or even based on semantic similarity.

[flagged]

Don't blame me, blame the people who invented the concept and wrote the original RAG paper.

https://arxiv.org/pdf/2005.11401

Re: Pruning RAG context down to what the answer actually needs

#45

Post by tingletech »

0x696C6961 wrote: post #13
tingletech wrote: post #6

Earlier quoted context omitted.

I read “RAG Context” as “the retrieved content injected into the context window”

So when an agent does "cat file.txt" that's RAG to you?

In the context of the title of the hn submission, I read “RAG Context” as “the retrieved content injected into the context window” .... "read" read /rε d/ not /riːd/

> So when an agent does "cat file.txt" that's RAG to you?

No, that might be "RAG Context" though.

Re: Pruning RAG context down to what the answer actually needs

#46

Post by dd8601fn »

jillesvangurp wrote: post #27
agentdev001 wrote: post #4

Am I wrong to be somewhat peeved by the use of "RAG" in these contexts? I always read things like this, and wonder if instead the author should be saying "Semantic Retrieval" or something something Vector, etc. Retrieval augmented generation captures tool-use, and; semantic search of course is really just a tool under the hood. To make an anology, in my mind, this is akin to saying "fuel air mixture system" when refe…

RAG is a fancy acronym that basically boils down to: let's give ai agents the super power of information retrieval (aka. search) and "augment" the generation with a list of results by adding that to the context. The narrow interpretation of this is usually some kind of vector search. Which some people naively treat as magic pixie dust that will make search quality amazing without any tuning whatsoever. This does not…

As a real dummy on the subject, maybe you could help me understand where vector search tends to fall over?

I use it to retrieve tool functions by description and it has worked very well for me, but I expect I'm in the "very simple use cases" category that you mentioned.

Re: Pruning RAG context down to what the answer actually needs

#49

Post by lbaltensp »

timedude wrote: post #47
Imanari wrote: post #40

Did you experiment with the Pruner completely replacing the Reranker?

Yep seems like the pruner and reranker could be combined. That is what i built.

Yes, the pruner operates on the top 15 chunks after reranking. reply
Post reply on HN