Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

twitter.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#51

Post by billmalarky »

oshams wrote: post #50
billmalarky wrote: post #49

Earlier quoted context omitted.

Do you build a syntax tree of the code, then loop the tree to auto-write an article for each node (or the larger or more material nodes) and then also reference the tree to pull in related nodes/pieces/modules/whatnot of the codebase when auto-writing a documentation article for each node? This is fascinating thank you

yeah ! you know the best way to learn more is to join us, we're hiring: https://www.workatastartup.com/companies/mutable-ai

Greatly appreciate the suggestion! but I recently joined another YC backed Gen AI startup in the fine-tuning space (OpenPipe) :-D

Speaking of, there's a good chance fine tuned models will be a component of your fully-optimized codebase -> wiki automation process at some point in the future. Likely to increase consistency/reliability of LLM responses as clear patterns start to emerge in the process. If y'all decide to layer that on or even explore it as an optimization strategy hit us (or me directly) up. We love collaborating w/ engineers working on problems at the edge like this, aside from how engaging the problems themselves are, it also helps us build our best possible product too!

Very excited to follow your journey! just sent you a LI request.

Thanks again so much for sharing your wisdom!

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#52

Post by oshams »

zellyn wrote: post #47
zellyn wrote: post #46

These wikis are really interesting. I'm itching to try it on the common framework parts of our work monorepo. [Update after looking at the Django wiki] The wiki's structure appears to be very (entirely?) based on the directory structure. Is that right? It would be interesting to give it the Django documentation in addition to the codebase, or possibly even just the heading structure of the documentation, and let it u…

Out of curiosity, would you mind pointing it at simonw/datasette? I think that might be interesting, particularly due to the plugin system.

sure thing, running it now!

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#53

Post by fudged71 »

spirobelv2 wrote: post #5

it wants me to login to ask a question I will just keep using phind you have vc dollars - sponsor a public free search over open source repos. Also think about what happens when your question touches multiple repos. I tried a similar "search github repo with ai" product before, but it led me right back to phind, when it couldnt answer a question that required specific information from the repositiory as well as a goo…

This comes across as pretty rude... they're already indexing and generating pages for public repos for free, and providing this search for free, just behind a login. If you want to use phind instead go ahead.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#54

Post by zby »

A prompt here is a textual presentation of a data structure - this is an area that I am exploring right now. I wonder if it would not be equally effective if the LLM was getting a JSON or Python representation instead of the presumed Markdown. But my intuition is that because LLMs are trained on texts that are meant for human consumption - then they will follow better exactly the same texts as humans find easier and that means that they 'want' exactly the same presentation as humans - nicely formatted text or nicely formatted programs, not mixes.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#55

Post by koliber »

PhilippGille wrote: post #13
langcss wrote: post #8

This sort of approach always made more sense to me than RAG. I am less likely to try RAG than something that feeds the LLM what it actually needs. RAG is risky in providing piecemeal information that confuses the LLM. The way I thought would work and like to try out is ask the LLM what info it wants next from an index of contents. Like a book. That index can be LLM generated or not. Then backtrack as you don't need t…

It's still RAG, just the R in RAG is not vector-based anymore, no?

You’re right. Many people ppl e take a mental shortcut and assume that RAG is a vector DB search. Any kind of retrieval is retrieval. You can do keyword search. You can do a PageRank like query. You can sort content by date and send the most recent items to the LLM. It’s all retrieval. That is the R on Retrieval Augmented Generation.

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#56

Post by oshams »

zellyn wrote: post #47
zellyn wrote: post #46

These wikis are really interesting. I'm itching to try it on the common framework parts of our work monorepo. [Update after looking at the Django wiki] The wiki's structure appears to be very (entirely?) based on the directory structure. Is that right? It would be interesting to give it the Django documentation in addition to the codebase, or possibly even just the heading structure of the documentation, and let it u…

Out of curiosity, would you mind pointing it at simonw/datasette? I think that might be interesting, particularly due to the plugin system.

hope you enjoy it ! https://wiki.mutable.ai/simonw/datasette

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#57

Post by paradite »

manishsharan wrote: post #30
anotherpaulg wrote: post #11

I agree that many AI coding tools have rushed to adopt naive RAG on code. Have you done any quantitative evaluation of your wiki style code summaries? My first impression is that they might be too wordy and not deliver valuable context in a token efficient way. Aider uses a repository map [0] to deliver code context. Relevant code is identified using a graph optimization on the repository's AST & call graph, not vect…

I tried using Aider but my codebase is a mix of Clojure Clojurescript and Java . I gave up making it work for me it as it created more issues for me. What I really hated about Aider was that it made code changes without my approval.

You can give 16x Prompt a try. It's GUI desktop app designed for AI coding workflow. It also doesn't automatically make code changes.

https://prompt.16x.engineer/

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#58

Post by abdullin »

maxsieg wrote: post #42

Hey, cofounder at Mutable.ai here. I want to encourage you all to ask the chat some tough questions. You can ask very complex and general questions. Some examples: - Ask ollama ( https://wiki.mutable.ai/ollama/ollama ) how to add a new model - Ask langchain ( https://wiki.mutable.ai/langchain-ai/langchain ) "How can I build a simple personal assistant using this repo?" - Ask flash attention ( https://wiki.mutable.ai/…

I’ve been building LLM-driven systems for customers for quite some time. We got tired of hallucinations from vector-based and hybrid RAGs last year, eventually arriving to the approach similar to yours.

It is even called Knowledge Mapping [1]. It works really well, and customers can understand it.

Probably the only difference with your approach is that we use different architectural patterns to map domain knowledge into bits of knowledge that LLMs will use to reason (Router, Knowledge Base, Search Scope, Workflows, Assistant etc)

My contacts are in the profile, if you want to bounce ideas!

[1] English article: https://www.trustbit.tech/en/wie-wir-mit-knowledge-maps-bess...

Re: Show HN: How we leapfrogged traditional vector based RAG with a 'language map'

#60

Post by spirobelv2 »

fudged71 wrote: post #53
spirobelv2 wrote: post #5

it wants me to login to ask a question I will just keep using phind you have vc dollars - sponsor a public free search over open source repos. Also think about what happens when your question touches multiple repos. I tried a similar "search github repo with ai" product before, but it led me right back to phind, when it couldnt answer a question that required specific information from the repositiory as well as a goo…

This comes across as pretty rude... they're already indexing and generating pages for public repos for free, and providing this search for free, just behind a login. If you want to use phind instead go ahead.

Thought show HN means being open for feedback not "please whorship my product"

here you go: your product is the best. You will come out on top in this very crowded market and your stocks will be worth millions.

I dont see it as rude to take my time and explain where their product is lacking. I am a power user of these kind of tools and tried many.

Post reply on HN