Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

twitter.com

Post reply on HN
Print view View original

11–20 of 62 posts

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#11

Post by anotherpaulg »

I agree that many AI coding tools have rushed to adopt naive RAG on code.

Have you done any quantitative evaluation of your wiki style code summaries? My first impression is that they might be too wordy and not deliver valuable context in a token efficient way.

Aider uses a repository map [0] to deliver code context. Relevant code is identified using a graph optimization on the repository's AST & call graph, not vector similarity as is typical with RAG. The repo map shows the selected code within its AST context.

Aider currently holds the 2nd highest score on the main SWE Bench [1], without doing any code RAG. So there is some evidence that the repo map is effective at helping the LLM understand large code bases.

[0] https://aider.chat/docs/repomap.html

[1] https://aider.chat/2024/06/02/main-swe-bench.html

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#13

Post by PhilippGille »

langcss wrote: post #8

This sort of approach always made more sense to me than RAG. I am less likely to try RAG than something that feeds the LLM what it actually needs. RAG is risky in providing piecemeal information that confuses the LLM. The way I thought would work and like to try out is ask the LLM what info it wants next from an index of contents. Like a book. That index can be LLM generated or not. Then backtrack as you don't need t…

It's still RAG, just the R in RAG is not vector-based anymore, no?

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#14

Post by TeMPOraL »

est wrote: post #12
dvh wrote: post #9

[flagged]

ahh, I remember someone on HN commented that screenshot is the defacto standard of "hypermedia"

Maybe you're mixing that with the observation that the Internet is mostly just some five image sharing sites, each posting screenshots of the other four.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#15

Post by lemming »

anotherpaulg wrote: post #11

I agree that many AI coding tools have rushed to adopt naive RAG on code. Have you done any quantitative evaluation of your wiki style code summaries? My first impression is that they might be too wordy and not deliver valuable context in a token efficient way. Aider uses a repository map [0] to deliver code context. Relevant code is identified using a graph optimization on the repository's AST & call graph, not vect…

I've been thinking about this a lot recently. So in Aider, it looks like "importance" is based on just the number of references to a particular file, is that right?

It seems like in a large repo, you'd want to have a summary of, say, each module, and what its main functions are, and allow the LLM to request repo maps of parts of the repo based on those summaries. e.g. in my website project, I have a documentation module, a client side module, a server side module, and a deployment module. It seems like it would be good for the AI to be able to determine that a particular request requires changes to the client and server parts, and just request those.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#16

Post by anotherpaulg »

lemming wrote: post #15
anotherpaulg wrote: post #11

I agree that many AI coding tools have rushed to adopt naive RAG on code. Have you done any quantitative evaluation of your wiki style code summaries? My first impression is that they might be too wordy and not deliver valuable context in a token efficient way. Aider uses a repository map [0] to deliver code context. Relevant code is identified using a graph optimization on the repository's AST & call graph, not vect…

I've been thinking about this a lot recently. So in Aider, it looks like "importance" is based on just the number of references to a particular file, is that right? It seems like in a large repo, you'd want to have a summary of, say, each module, and what its main functions are, and allow the LLM to request repo maps of parts of the repo based on those summaries. e.g. in my website project, I have a documentation mod…

The repo map is computed dynamically, based on the current contents of the coding chat. So "importance" is relative to that, and will pull out the parts of each file which are most relevant to the task at hand.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#17

Post by zbyforgotp »

PhilippGille wrote: post #13
langcss wrote: post #8

This sort of approach always made more sense to me than RAG. I am less likely to try RAG than something that feeds the LLM what it actually needs. RAG is risky in providing piecemeal information that confuses the LLM. The way I thought would work and like to try out is ask the LLM what info it wants next from an index of contents. Like a book. That index can be LLM generated or not. Then backtrack as you don't need t…

It's still RAG, just the R in RAG is not vector-based anymore, no?

What you describe sounds like Agetic RAG https://zzbbyy.substack.com/p/agentic-rag

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#18

Post by lemming »

anotherpaulg wrote: post #16
lemming wrote: post #15

Earlier quoted context omitted.

I've been thinking about this a lot recently. So in Aider, it looks like "importance" is based on just the number of references to a particular file, is that right? It seems like in a large repo, you'd want to have a summary of, say, each module, and what its main functions are, and allow the LLM to request repo maps of parts of the repo based on those summaries. e.g. in my website project, I have a documentation mod…

The repo map is computed dynamically, based on the current contents of the coding chat. So "importance" is relative to that, and will pull out the parts of each file which are most relevant to the task at hand.

Interesting, how does Aider decide what’s relevant to the chat?

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#19

Post by janalsncm »

zbyforgotp wrote: post #17
PhilippGille wrote: post #13

Earlier quoted context omitted.

It's still RAG, just the R in RAG is not vector-based anymore, no?

What you describe sounds like Agetic RAG https://zzbbyy.substack.com/p/agentic-rag

> The traditional way to do RAG is to find information relevant to a query - and then incorporate it into the LLM prompt together with the question we want it to answer.

Technically this is incorrect. The original RAG paper used a seq2seq generator (BART) and involved two methods: RAG sequence and RAG token.

RAG sequence used the same fixed documents and appended them to the input query (note, this is different from a decoder-only model). RAG token generates each token based on a different document.

I only nitpick this because if someone is going to invent new fancy-sounding variants of RAG they should at least get the basics right.

Post reply on HN